What is a Real-Time Database?
A real-time database is a database system able to handle data workloads that are in constant flux and are extremely time-sensitive. Unlike traditional databases, which contain data that is persistent and changes much less frequently.
Real-time processing means that transactions are processed very quickly, enabling the organization or other integrated systems to act on the data immediately. Real-time databases are typically used for accounting, banking, legal, medical records, multimedia, process control, reservation systems, and scientific data analysis.
- How Does a Real-Time Database Work?
- 9 Characteristics of Live Real-Time Databases
- Real-Time Databases Example Applications
- Raima Real-Time Database Solution
How Does a Real-Time Database Work?
A real-time database, also known as a live database, is a traditional database with extended performance and throughput capabilities. They apply time constraints to data, to indicate at which point in time the data is still valid (also known as temporal validity). In many cases, a sensor or feed from the outside world constantly updates the data with new, currently relevant data.
A real-time database must meet several strict criteria:
- Handle queries that are extremely time-sensitive
- Only return temporally valid data
- Support priority scheduling for queries and responses
In a real-time database system, there is a significant concern as to how to record the exact time at which data was recorded and queries were received-detailed timestamps provide a clear time reference. Based on these timestamps, the system must ensure real-time database queries are processed consistently under the above constraints.
A live database must also address a situation when a real-time database query cannot be answered fast enough. For example, if data is provided late, it can be provided with a flag indicating the time of the response, so the operator can consider if the response is still relevant or not.
Research and development of real-time databases are likely to increase over the coming years, with the exponential growth in real-time applications. Video calling is now ubiquitous around the world, messaging and social applications like WhatsApp and Facebook have billions of users, and there is a growing demand for live streaming of high-definition video and audio, which has real-time properties. New protocols and technologies will emerge to process real-time transactions more efficiently.
9 Characteristics of Live Realtime Databases
The burden on software in a “live” information system is simply to keep up. Processing must occur at a rate faster than the activity. It must represent the state of a system within a certain time tolerance. A stock quote should be only a few seconds old, but a temperature reading from an elementary school can be 15 minutes old. Both are considered “real-time” data.
Database systems that satisfy determinism requirements for hard real-time systems usually need to be simplified versions, where queries and updates are simple and bounded. While technology and algorithms are continually improving in this area, there are still only a few database systems that truly meet hard real-time requirements.
1. Performance
“Keeping up” with live activity is the requirement of the database system together with the software that captures and/or displays the data. When there are 100 events/minute, and the system can handle 1000 events/minute, then everything’s fine. But what if events average 2000/minute with bursts of over 10,000? In this case, a system that is three times faster is required. Higher-performing database management systems will be able to solve more real-time problems.
2. Historical Records
If sensors are being read and a representation of the data is being displayed, there may be no need for a database. However, if the live data display must be logged so that historical views or summaries can be produced, then the database must be able to capture and store this data in a timely way that facilitates future access.
3. Maintain Relationships
Monitoring live data frequently involves raising alarms or triggering additional actions when certain conditions are met. Those conditions may involve the presence or absence of data in the database, so quick lookups must be performed. They may also depend on connections between records in the database. The database system is required to maintain associations and lookups that can be quickly created or queried.
4. Queuing
Assuming that inputs into a live system are irregular, as they often are, the system must handle bursts of events that can be processed soon, where “soon” is during a quieter period. Then it is likely that processing the events in order is important. So a queuing model that maintains order in the backlog will satisfy this requirement.
5. Durability
Again, assuming that the data is not transient, the database system is present in order to keep the data safe, which normally means getting it onto a disk drive or flash drive. The best systems will do this in a way that is recoverable, meaning that a computer crash will not cause any corruption of the database.
6. Batching
Transactions that write data to a durable media are the most time-consuming function in a database system. It may not always be important to perform one transaction for each event. When this is the case, one transaction may commit 2 to 200 events at once, meaning that the database system is using far fewer resources and thus performing faster. Batching depends on the tolerance for loss of data (in the case of a crash) and the freshness requirements of the data. For example, if a display of the live data is updated every five seconds, it makes no difference if the 300 events that occurred within the last five seconds were committed with 300 transactions of 1 event each, or 3 transactions of 100 each.
7. Conditioning Data
One technique to allow greater volumes of data to be archived is to reduce, or “condition” the data before storing it. This may involve filters that ignore parts of it, or translations that reduce larger representations into smaller ones. Both of these techniques may require fast indexing of previously stored data.
8. Non-interfering Queries
If a system is strained to capture and store event data, it may be slowed down even more when the data must be read for display purposes. Techniques for reading data without locks are needed to prevent this. Two common techniques are known as MVCC and dirty reading. MVCC stands for “Multi-Version Concurrency Control” and creates a virtual snapshot of a database so that a consistent query can be made. Dirty reading is even faster when it is not necessary to rely upon the consistency of relationships between records in a database.
9. Aggregation
Another common computing model has multiple “collector” computers that capture and store event data locally, then pass it on to an aggregation site. The common technology to solve this problem is replication. This means that the changes to the local databases are also made to the aggregate database. This usually implies that the computers are connected through networking.
Meaningful handling of live, time-sensitive data is the new real-time. The old hard real-time systems are still very important and pervasive, but cannot use many of today’s open-source or commercial database management systems.
Real-Time Databases Example Applications
Real-time applications (RTAs) are applications that operate within the immediate or current time frame of user searches. The delay must be less than a predefined value.
RTAs are evaluated using a metric called worst-case execution time (WCET)-this is the maximum time allowed for a task, given a specific hardware configuration. Applications with a WCET of several seconds or less are considered real-time. The process of operating under real-time constraints is known as real-time computing (RTC).
Here are a few examples of RTAs:
- Voice over Internet Protocol (VoIP)
- Messaging, chatting, and instant message (IM) applications
- Video conference
- Online gaming
- Stock trading
- E-commerce transactions with multiple bidders
Raima Real-Time Database Solution
Raima Database Manager (or RDM) is an ACID-compliant embedded database management system designed for use in embedded systems applications. RDM has been designed to utilize multi-core computers, networking (local or wide area), and on-disk or in-memory storage management. RDM provides support for multiple application programming interfaces (APIs): low-level C API, C++, and SQL(native, ODBC, JDBC, ADO.NET, and REST). RDM is highly portable and is available on Windows, Linux, Unix, and several real-time or embedded operating systems. A source-code license is also available.
RDM has support for both non-SQL (record and cursor level database access) and SQL database design and manipulation capabilities. The non-SQL features are important for the most resource-restricted embedded system environments, where high performance in a very small footprint is the priority. SQL is important in providing a widely known standard database access method in a small enough footprint for most embedded systems environments.
