What is an in-memory database management system?
An In-Memory database is stored in a computer’s main memory (RAM), and is managed by an In-Memory Database Management System. Traditional databases are stored on disk drives.
Traditional disk-based databases are formatted with an awareness of the block-oriented device on which the data is read and written. When one part of a database refers to another part, it may require a different block to be read from the disk. This is a non-issue with in-memory databases, and interrelationships between parts of the database can be managed through direct pointers. The in-memory data is always present so there is no latency for reading. Writing data to disk must be done in an “atomic” fashion, where all writes are registered, or none of them are. This often requires journaling or double-writes. In-memory has no such requirement, so the changes to memory are nearly instantaneous.
There is a trade-off between in-memory and on-disk databases, where in-memory data can be accessed and updated much more quickly than on-disk data, but the price of the great performance is an increased risk of data loss.
Computer systems are more reliable than ever, so this risk is decreasing. If a computer loses power, it may cause the loss of any unsaved data. It is an administrative decision to set the amount of potentially lost data. In effect, an in-memory database has “save points” that are more granular than transactions in an on-disk database. If the in-memory data is saved to disk after each transaction, it is the same as an on-disk database. However, the in-memory database may be saved with different criteria. For example, after every 100 rows are added, or every 1 minute, or every day. This allows control over the performance and risk.
How is data accessed and changed in an in-memory database management system?
Data in an RDM in-memory database is “ready to go.” Where data in a disk block may be compressed, encrypted, flattened or encoded, data in memory is in a directly usable format. It also has structure that is independent of disk blocking issues, allowing direct navigation from column to column or row to row or index to row, and also allowing changes to be implemented by allocating memory blocks and re-arranging pointers.
The following figure contrasts this difference, showing how in-memory databases shortcut the process of preparing data for use.
Another important performance factor is that access to data that resides in the process space of an application requires no calls into the operating system. Traditional operating systems will “switch context” from one application to another when they gain control. When disk reads and writes are avoided, the application can stay in control of the CPU for longer periods of time, which is especially important in real-time applications.
How is in-memory data shared among multiple tasks?
The answer is two-fold: fast and faster. For those familiar with the RDM Transactional File Server, which can be used to share a database among user processes in the same computer, in the office LAN, or across the world, the database can be managed in-memory by the TFS. This is faster than on-disk databases managed by the TFS.
But the fastest way to share one database among multiple tasks is to run a single process with multiple threads, where each task is running in its own thread. This form of an application runs on one computer and allows each thread to access the database directly in local heap storage. It also circumvents the need to read unchanged database objects into local cache, because the original object can be viewed directly from the database memory. This is a very fast way to share a database in a multi-user mode.
Can an on-disk database be used in-memory?
When using RDM, this answer is “Yes!” In-memory databases may be loaded from disk when they are opened, and may also be saved to disk when they are closed, or when “save points” are requested. Save points are atomic, meaning that all changes to the data since the last save point are written to disk together, or none of them are. This makes them transactional, but not on the same transaction boundaries as an on-disk database.
What is under the covers of Raima’s current technical solutions? What is a database union? How does MVCC impact read performance?
Go to current Technology Page
Raima’s collection of informative and instructional videos are available on one page.
Go to Videos
Raima’s Development and QA team occasionally add details about topics not found elsewhere.
Go to Blog page