Vector brain symbolising in memory

In-Memory Database

Raima Database Manager (RDM) is an in-memory database (IMDB) that allows applications to store the database in main memory. With Raima´s support of the next generation OLTP edge applications for real-time data, our clients are ensured a scalable and performing solution.

What is an in-memory database management system?
An in-memory database is stored in a computer’s main memory (RAM), and is managed by an in-memory database management system. Traditional databases are stored on disk drives.

Traditional disk-based databases are formatted with an awareness of the block-oriented device on which the data is read and written. When one part of a database refers to another part, it may require a different block to be read from the disk. This is a non-issue with in-memory databases, and interrelationships between parts of the database can be managed through direct pointers. The in-memory data is always present, so there is no latency for reading. Writing data to disk must be done in an “atomic” fashion, where all writes are registered, or none of them are. This often requires journaling or double-writes. An in-memory database has no such requirement, so the changes to memory are nearly instantaneous.

How is data accessed and changed in an in-memory database management system?
Data in an RDM in-memory database is “ready to go.” Where data in a disk block may be compressed, encrypted, flattened or encoded, data in memory is in a directly usable format. It also has structure that is independent of disk blocking issues, allowing direct navigation from column to column, row to row or index to row, allowing changes to be implemented by allocating memory blocks and rearranging pointers.

Can an on-disk database be used in-memory?
When using RDM, the answer is “Yes!” In-memory databases may be loaded from disk when they are opened, and may also be saved to disk when they are closed, or when “save points” are requested. Save points are atomic, meaning that all changes to the data since the last save point are written to disk together, or none of them are. This makes them transactional, but not on the same transaction boundaries as an on-disk database.

Raima Embedded In memory database

How is in-memory data shared among multiple tasks?
The answer is twofold: fast and faster. For those familiar with the RDM Transactional File Server (TFS), which can be used to share a database among user processes in the same computer, in the office LAN or across the world, the database can be managed in-memory by the TFS. This is faster than on-disk databases managed by the TFS.

But the fastest way to share one database among multiple tasks is to run a single process with multiple threads, where each task is running in its own thread. This form of an application runs on one computer and allows each thread to access the database directly in local heap storage. It also circumvents the need to read unchanged database objects into a local cache, because the original object can be viewed directly from the database memory. This is a very fast way to share a database in a multi-user mode.

How does Raima implement an in-memory database system in RDM?

The Raima RDM database engine is divided into two components: the runtime engine and the storage engine.
The runtime is responsible for transaction management, rule enforcement, query processing, and maintaining local, uncommitted, changes to the database. Whenever the runtime engine needs a database object, it requests the object from the storage engine. In addition, when local changes in the runtime are committed, they are given to the storage engine to be stored in a location shared between all users of the database. The runtime engine does not know or care if the storage engine is using main memory or secondary storage.

The storage engine can be thought of as a key/value pair repository. The keys are database object identifiers and the values are the location of the database objects. For disk-based databases, the values will be a file identifier, file offset, and size, for in-memory database the values will be the memory location of the database object.
The objects themselves may be encoded, compressed, and encrypted, but the runtime engine will know how to interpret the contents.

By maintaining this abstraction, an application developer doesn’t need to do anything other than specifying a pre-open configuration parameter to use an in-memory RDM database. The application, runtime, and storage engine can all run in the same memory space providing very low latency access. If a developer knows that a database will be primarily used in-memory they can utilize methods such as AVL indexing to optimize for in-memory access.

Read more about Raima´s in-memory implementation.

Learn about how Raima provides high-performance, portable embedded database and mobile database solutions.