As an ever-growing number of smart, connected devices generate more and more data in a global information space, Raima´s David Nguyen, Director of Engineering looks at how embedded database technology will underpin the Internet of Things.
IDC estimates IoT hardware to be the largest technology category in 2018 with $239 billion going largely toward modules and sensors along with some spending on infrastructure and security. Furthermore IDC expects “By 2021, more than 55% of spending on IoT projects will be for software and services.
Where is all this data coming from? Increasingly, it is being driven by ever-smarter devices, appliances and machines communicating over the internet. It is believed there will soon be more ‘things’ than people connected across cyberspace, and this ‘Internet of Things’ will drive tremendous changes in the way we live and work. Notably, we will see intelligent systems making autonomous decisions.
We can consider two vital aspects of the Internet of Things. The first is the communications infrastructure itself. Where once communications between devices was expensive and data-limited, today the likes of GSM technology (5G) and the ubiquity of Ethernet have removed those restrictions and opened up a new paradigm of machine-to-machine (M2M) communications. With this enabling technology now something we can take for granted, the second aspect has come sharply into focus: how to manage the large amounts of data being generated on the vast numbers of different products and systems.
There is no doubt that the Internet of Things represents a significant challenge for data management and analysis. And as the volume of data being generated grows exponentially, the problem can only become more acute. If we don’t address the challenge with appropriate data management tools, then the risks are: on the one hand that our traditional information management systems become simply a dumping ground for data that appears more quickly than it can ever be analysed or used; and on the other hand that local devices will never realise their potential.
We also have to consider that, in this new world of information, data can come in many different forms – numbers, text, images, audio, video – and from many different sources. In the world of manufacturing, not only will this mean increasing data from sensors, actuators, control systems and smart cameras, but also from the actual products being manufactured – a key aspect that is driving the concept of either Industrial Internet of things or ‘Industry 4.0’. When devices can make their own decisions as they move through the manufacturing process, then there is vastly improved the potential for optimizing productivity.
But there is more, because smart mobile devices are also becoming an increasingly important part of the manufacturing landscape, as people turn to tablets and smart phones for plant monitoring and asset management.
Similar patterns of device communication are emerging in other fields of industry, and the Internet of Things is even seen as a major driver for change in our home lives. An example of this is the often quoted intelligent refrigerator that will monitor food stocks, communicate directly with supermarkets when we are getting low on particular products, and will text menu suggestions to us when we arrive home based on the contents of the fridge and the use-by dates of particular items.
With so many new sources of data, distributed across so many nodes, and with computing power and storage capacity distributed across huge numbers of devices, the challenge of managing this data is certainly acute. Perhaps more than ever before, we have to look at issues of availability, consistency, reliability and performance, and traditional database systems quickly fall short of our requirements.
The alternative is embedded database technology, operating on different hardware and software platforms, providing local data management and data distribution capability, safely, efficiently and securely. These embedded systems give local applications the intelligence to analyse and distribute data, make decisions autonomously, or summarise data efficiently for other systems.
The embedded database industry has responded to the requirements of the Internet of Things with data management products that deliver the requisite performance and availability in products that are readily scalable. These data management products can take the captured live data, process it (aggregating and simplifying the data as required) and then distribute it to deliver the visualization and analytics that will enable meaningful decisions to be made.
Typically, embedded database technology products are cross-platform, small footprint, fast and reliable persistent, in-memory and hybrid database solutions which are optimized for workgroup, live real-time, embedded and mobile operating systems. They are designed for distributed architectures in resource-constrained environments, and developed to fully utilize multi-core processors. Importantly, they are suitable for running on a wide variety of platforms, and support multiple APIs and configurations which provide developers with numerous powerful programming options and functionality.
The objective is that embedded databases provide rugged, scalable and local solutions for the handling of large amounts of data at any time, locally. Platform independence means that they can run on everything from popular OS options such as MS Windows, Linux and iOS to customized and bespoke systems. A data storage engine is used to control in-memory, disk-based or remote storage to provide the best possible performance.
With an embedded database information can be stored locally, and queried efficiently with low overhead APIs, while protecting data and providing vastly improved stability on resource-constrained systems compared with conventional database solutions.
Embedded solutions collect and store data in a highly structured way, which allows pre-processing of the data actually on the embedded device itself before sending the most relevant data to other systems for further analysis or long-term storage.
Importantly, embedded technology can makes data available wherever it is needed within the system and, via the Internet, to the wider environment.
With Today’s embedded capabilities, system developers across all sectors can take full advantage of features such as support for multi-core processors, in-memory limitation, encryption, shared memory and a host of other high-performance features. ACID compliance means that the information collected is guaranteed to be accurate.
There is no doubt that the Internet of Things will drive a new paradigm in distributed intelligence and autonomous operation, but to deliver on the promise of this emerging technology we need robust, connected applications with data management tools that will cope with the huge, dynamic and rapidly expanding network of smart devices that are simultaneously communicating with each other and generating data. Embedded database technology can address the challenge of managing these high levels of data, efficiently, effectively, securely and in real-time.
