This is an introduction to RDM’s Core Database Engine. At the core, it securely stores, collects and manages data. The SQL engine, that sits on top of the core, provides a standards approach to programming on the core engine.
What is the Core Database Engine?
Beneath all the shiny SELECT statements, chrome UPDATE statements, and flashy INSERT statements, a database is essentially just a way of storing data in on a persistent data store in a structured and safe way. A SQL engine can make that data simpler to handle allowing English-like statements to be used to create, modify, delete, and request data. Once a SQL statement has been parsed down so the engine knows what to do, some portion of the Database Management System (DBMS) has to know how to read from and write to the physical files.
In the RDM product family this layer is referred to as the Core engine. Historically this piece of the system arrived long before the SQL database engine, as early customers were all C programmers using structures and managing all their queries themselves. When the SQL engine came along, there was no reason to reinvent the capabilities already present in the Core engine, so the SQL database engine sits on top. Each SQL statement is parsed into the information needed to tell the Core engine what to do.
The Core engine, therefore contains all the capabilities to create new records (or rows in SQL terms), update existing records, delete records, find records, and navigate from one record to another. In addition it contains all the capabilities to manage different sessions, cache the files, handle transactions (grouping changes together into one logical unit), protect access to records via locking, etc. The Core database engine is the guts of the RDM Database Management System.
Core Database Engine Pieces
Prior to version 10.0, the Core engine was essentially a monolith – everything was in the application’s process space – with the exception of locking which was handled by a separate lock manager application. Starting in version 10.0 the Core engine was split into two components – the runtime engine and the TFS (transactional file system). The TFS encapsulates all of the transaction and file handling abilities, along with the locking from the original lock manager application, but it doesn’t know anything about the structure of the data. The runtime on the other hand, knows everything about the structure of the data, but relies on the TFS to get data from and put data into the database files.
Think of the world-wide shipping industry. It used to be that every item shipped had to be managed separately. But then, along came shipping containers of standard sizes and cargo ships, trains and trucks that could manage shipping containers and things got much more efficient. The TFS is like the ships, trains, and trucks. It takes whole pages of data (like shipping containers) and handles them, never needing to know what it inside, while the runtime is the part of the code that puts the individual data records and fields into those pages (packing a shipping container).
Different TFS flavors
The runtime engine then handles the requests from the user, dealing with the specific data that the user needs, getting pages from and handing pages to the TFS. This provides for the ability to have multiple different TFS types with different capabilities.
Currently, the RDM product family supports three different TFS versions: called TFST, TFSR, and TFSS. Broadly speaking TFST is full transactional functionality in the same process as the runtime, TFSR is full transactional functionality but with the TFS functionality residing in a separate process, and TFSS is reduced functionality (and therefore faster) and is designed for single-user access.
The TFST version contains all the main functionality including supporting transactions and locking. It is designed for situations where a single application (possibly with multiple threads) is accessing the database. For those situations where there are multiple applications that need to access the database simultaneously, the TFSR can be used. In this case a central tfserver process is run (which contains a copy of the TFST) and the TFSR that is used by the application sends its requests over to this tfserver for processing. The TFSS option is useful for simple applications that need single-user access to the database without full-transactional support.
Conclusion
The SQL database engine for RDM is built on top of a full featured, configurable Core engine. This engine can be configured depending on the needs of the specific application.
