9 Attributes of Live Real Time Databases

January 15

Raima logo on grey background

What is a Real Time Database?

A real-time database is a database system able to handle data workloads that are in constant flux and are extremely time-sensitive. Unlike traditional databases, which contain data that is persistent and changes much less frequently. 

Real-time processing means that transactions are processed very quickly, enabling the organization or other integrated systems to act on the data immediately. Real-time databases are typically used for accounting, banking, legal, medical records, multimedia, process control, reservation systems and scientific data analysis.


 

How Does a Real Time Database Work?

A real time database, also known as a live database, are traditional databases with extended performance and throughput capabilities. They apply time constraints to data, to indicate at which point in time the data is still valid (also known as temporal validity). In many cases, a sensor or feed from the outside world constantly updates the data with new, currently relevant data.

A real time database must meet several strict criteria:

  • Handle queries that are extremely time-sensitive
  • Only return temporally valid data
  • Support priority scheduling for queries and responses

In a real time database system, there is a significant concern as to how to record the exact time at which data was recorded and queries were received-detailed timestamps provide a clear time reference. Based on these timestamps, the system must ensure realtime database queries are processed consistently under the above constraints. 

A live database must also address a situation when a real time database query cannot be answered fast enough. For example, if data is provided late, it can be provided with a flag indicating the time of the response, so the operator can consider if the response is still relevant or not.

Research and development of real time databases is likely to increase over the coming years, with the exponential growth in real time applications. Video calling is now ubiquitous around the world, messaging and social applications like WhatsApp and Facebook have billions of users, and there is a growing demand for live streaming of high definition video and audio, which has real time properties. New protocols and technologies will emerge to process real-time transactions more efficiently.

 

9 Characteristics of Live Realtime Databases

The burden on software in a “live” information system is simply to keep up. Processing must occur at a rate faster than the activity. It must represent the state of a system within a certain time tolerance. A stock quote should be only a few seconds old, but a temperature reading from an elementary school can be 15 minutes old. Both are considered “real time” data.

Database systems that satisfy determinism requirements for hard real time systems usually need to be simplified versions, where queries and updates are simple and bounded. While technology and algorithms are continually improving in this area, there are still only a few database systems that truly meet hard real time requirements.

1. Performance

“Keeping up” with live activity is the requirement of the database system together with the software that captures and/or displays the data. When there are 100 events/minute, and the system can handle 1000 events/minute, then everything’s fine. But what if events average 2000/minute with bursts of over 10,000? In this case, a system that is three times faster is required. Higher performing database management systems will be able to solve more real time problems.

2. Historical Records

If sensors are being read and a representation of the data is being displayed, there may be no need for a database. However, if the live data display must be logged so that historical views or summaries can be produced, then the database must be able to capture and store this data in a timely way that facilitates future access.

3. Maintain Relationships

Monitoring live data frequently involves raising alarms or triggering additional actions when certain conditions are met. Those conditions may involve the presence or absence of data in the database, so quick lookups must be performed. They may also depend on connections between records in the database. The database system is required to maintain associations and lookups that can be quickly created or queried.

4. Queuing

Assuming that inputs into a live system are irregular, as they often are, the system must handle bursts of events that can be processed soon, where “soon” is during a quieter period. Then it is likely that processing the events in order is important. So a queuing model that maintains order in the backlog will satisfy this requirement.

5. Durability

Again, assuming that the data is not transient, the database system is present in order to keep the data safe, which normally means getting it onto a disk drive or flash drive. The best systems will do this in a way that is recoverable, meaning that a computer crash will not cause any corruption of the database.

6. Batching

Transactions that write data to a durable media are the most time-consuming function in a database system. It may not always be important to perform one transaction for each event. When this is the case, one transaction may commit 2 to 200 events at once, meaning that the database system is using far fewer resources and thus performing faster. Batching depends on the tolerance for loss of data (in the case of a crash) and the freshness requirements of the data. For example, if a display of the live data is updated every five seconds, it makes no difference if the 300 events that occurred within the last five seconds were committed with 300 transactions of 1 event each, or 3 transactions of 100 each.

7. Conditioning Data

One technique to allow greater volumes of data to be archived is to reduce, or “condition” the data before storing it. This may involve filters that ignore parts of it, or translations that reduce larger representations into smaller ones. Both of these techniques may require fast indexing of previously stored data.

8. Non-interfering Queries

If a system is strained to capture and store event data, it may be slowed down even more when the data must be read for display purposes. Techniques for reading data without locks are needed to prevent this. Two common techniques are known as MVCC and dirty-reading. MVCC stands for “Multi Version Concurrency Control” and creates a virtual snapshot of a database so that a consistent query can be made. Dirty reading is even faster when it is not necessary to rely upon the consistency of relationships between records in a database.

9. Aggregation

Another common computing model has multiple “collector” computers that capture and store event data locally, then pass it on to an aggregation site. The common technology to solve this problem is replication. This means that the changes to the local databases are also made to the aggregate database. This usually implies that the computers are connected through networking.

Meaningful handling of live, time-sensitive data is the new real time. The old hard real time systems are still very important and pervasive, but cannot use many of today’s open source or commercial database management systems.

 

Real Time Databases Example Applications 

Real-time applications (RTAs) are applications that operate within the immediate or current time frame of user searches. The delay must be less than a predefined value. 

RTAs are evaluated using a metric called worst-case execution time (WCET)-this is the maximum time allowed for a task, given a specific hardware configuration. Applications with a WCET of several seconds or less are considered real time. The process of operating under real time constraints is known as real-time computing (RTC).

Here are a few examples of RTAs:

  • Voice over Internet Protocol (VoIP)
  • Messaging, chatting and instant message (IM) applications
  • Video conference
  • Online gaming
  • Stock trading
  • Ecommerce transactions with multiple bidders

 

Raima Real Time Database Solution

Raima Database Manager (or RDM) is an ACID-compliant embedded database management system designed for use in embedded systems applications. RDM has been designed to utilize multi-core computers, networking (local or wide area), and on-disk or in-memory storage management. RDM provides support for multiple application programming interfaces (APIs): low-level C API, C++, and SQL(native, ODBC, JDBC, ADO.NET, and REST). RDM is highly portable and is available on Windows, Linux, Unix and several real-time or embedded operating systems. A source-code license is also available.

RDM has support for both non-SQL (record and cursor level database access) and SQL database design and manipulation capabilities. The non-SQL features are important for the most resource-restricted embedded system environments, where high performance in a very small footprint is the priority. SQL is important in providing a widely known standard database access method in a small enough footprint for most embedded systems environments.

 

Get notified about new RDM updates

Be the first to know about new Raima Database Manager updates when they go live, use cases, industry trends and more.