Lead Engineer Jeff Parsons discusses the advantages and disadvantages of in-memory databases. Jeff uses the ACID principles as the core for his comparison. Also learn what Raima provides as a data management in-memory solution.
Transcripts
I would like to discuss the advantages together with the disadvantages of what we call “in memory databases“. First of all, what is the difference between an in-memory database and simply storing data in shared memory segments? Traditionally, databases can be characterized by the four attributes commonly referred to by the acronym ACID: Atomicity, Consistency, Isolation and Durability. As soon as you stop persisting a database, you lose durability. So this is what is lost with an in-memory database. Aside from methods of copying database contents to another durable location, an in-memory database can be subject to data loss if something stops working.
The remaining attributes; atomicity, consistency and isolation are what differentiate an in-memory database from simple data held in some shared memory. Atomicity is the word used to describe transactional behavior where all changes are committed to the database as an all or nothing operation. Isolation is the word used to describe what transactional changes other processes see. They can never be allowed to see partial transactions. In its most basic form, shared memory does not enforce atomicity or isolation. That is fine as long as there is only one process using the data, but that defeats the purpose for shared memory.
Database consistency means that structural rules and relationships are maintained for all users. Enforcement of database consistency requires sharing common code and data definitions. Of course, this can be done with shared memory, but a database system will have code and data definition built-in and ready to use. So, regardless of whether your data is durable or in memory, you still need database functionality if your data is shared or complicated, and that is not easy to do yourself.
The main reason to keep a database in-memory is performance. Reading and writing data that is purely in memory is faster than data stored on disk or on a flash drive. The corresponding disadvantage is that all data must fit in memory. This is increasingly affordable, but memory sizes are still multi-gigabyte, while disk size can be multi-terabyte. It is necessary to buy adequate memory or limit the size the database. How does Raima implement an in memory database system in RDM? We allow you to declare an entire database or a selection of tables in the database as in memory. That’s all you have to do. We implement it through our transactional file server and runtime library.
The transactional file server is an independent process that controls the database files and maintains atomicity and isolation for all users on the same computer or others. The runtime library is linked into the application code. It writes and reads database pages to and from the TFS. The key is, when a database or a portion of a database is declared as in-memory, only the RDM TFS and runtime behave any differently. Your application code can remain exactly the same. RDM has several options for in memory databases.
A persistent in memory database is loaded from disk image when it is opened and changes are automatically saved back when it is closed. A volatile in memory database is empty when it is open and all data is discarded when it is closed. The third type is a read on open, which automatically loads data from a disk image when opened, but changes are only written on demand. So, to summarize, the advantages of using RDM as an in memory database, is that it provides transactional, multi-user safety where a shared memory solution requires a lot of work to do this. It also provides greater performance than durable solutions.
Disadvantages; potential loss of data and limit on database size. When this is fine, you should certainly use RDM’s in memory solution.
