Bill talks about the Driving Factors for Change in the Embedded Database Industry and relates this to SQL, Cloud Computing, Performance as a driving factor and more.
According to Agrawal, et al. (2009, June) The database community is at a turning point in its history, due to both an explosion of data and usage scenarios and major shifts in computing hardware and platforms.
Driving Factors for Change in the Embedded Database Industry
A meeting of database researchers, architects, users and pundits discussed several factors driving changes in the database industry. Some of those factors are: excitement over “big data”; data analysis as a profit center; ubiquity of structured and unstructured data; expanded developer demands; and, architectural shifts in computing. The question that I struggle with is how these major shifts in the database community affect the embedded computing industry. It is an absolute that there is and will be an affect.
Impact of Data Analysis
We have seen the impact of the excitement over the “data analysis” factor in some of our customers. As Agrawal, et al. (2009, June) states, “Statistically sophisticated analysts are being hired in a growing number of industries, with increasing interest in running their formulae on the raw data.” In other words, these analysts are not only interested in the production line quotas, but may also be interested in accessing the data generated by the myriad of automated devices along the production line. The raw data may hold the key to increased efficiency and profit for the company and access to it is becoming a necessity.
Expanded Developer Demands
One of the most complex factors for me to generalize is the “expanded developer demands”. While there has been a significant rise in the adoption of relational DBMSs in the application developer community, expanded requirements of the industries needing data management are creating new expectations for programmability and usability. SQL, while exceptionally good for many types of database queries may be overly cumbersome in some applications. A new programming model with simplified system components for data management and manipulation may be more easily accepted than SQL in some of these application types. In other words, application developers are looking for the fastest way possible to develop their applications with quality. In some cases, SQL is stated as a requirement because it is considered a standard and someone on the team knows it. The real requirement is they want to capture and store information quickly and any API that provides that ability with the fewest number of lines of code, the better.
Performance and Driving Factors | Embedded Database Industry
The driving factors for change in the embedded database industry can be traced back to performance. Larger data sets mean the engine needs to scale to handle searches efficiently. Time-to-market is another performance indicator which is dictated by ease-of-use and interoperability with other application development tools. In the embedded systems space, in memory database performance or the ability to provide deterministic response times, especially in light of analysts needing access to the device data is a priority.
When it comes to increased performance, Jouppi (2009, June) reports, “The long march of single-thread performance increases has stopped in its tracks.” Instead of faster processors, we are now seeing more processor cores per die. Recent design changes to the Raima product line are focusing on the behavioral characteristics of these processors in order to provide true parallel processing. The more work that can be done in parallel without interaction or synchronization with other threads or processes will significantly increase the throughput of the embedded database engine.
The internet is another area where increased bandwidth is limited by the laws of physics. The advent of “cloud computing” will be sensible for many types of applications. Smith (2009, July) states, “As [cloud computing] grows in popularity, engineers should be aware of some fundamental limitations they face when developing these kinds of applications-in particular, the finite speed of light”. For many other applications, the lack of deterministic response times will prevent the application functionality from being adopted. However, a mixture of embedded database applications running on the edge of the “cloud” would be a likely scenario to preserving deterministic response times with the ability to interoperate with the “cloud” for some of these applications.
From my perspective, the market seems to be on the verge of fragmenting even more. Some of the givens for the next period are:
- SQL will grow as a requirement, but not in all segments
- Usability and time-to-market will be very high on developer requirements, but not necessarily SQL
- The need for interoperability with access and movement of data toward analysis centers (could be “cloud” based)
- Performance will be an absolute requirement
- Typical ACID database requirements may not be a strict requirement for all application use cases.
Based on my reading over the last few days, the RDM Embedded project seems to be poised to take advantage of following the further fragmentation of the embedded database industry.
References:
AGRAWAL, R., AILAMAKI, A., BERNSTEIN, P., BREWER, E., CAREY, M., CHAUDHURI, S., et al. (2009, June). The Claremont Report on Database Research. Communications of the ACM, 52(6), 56-65. Retrieved July 14, 2009, from Business Source Alumni Edition database.
Jouppi, N. (2009, June). Technical Perspective: Software and Hardware Support for Deterministic Replay of Parallel Programs. Communications of the ACM, 52(6), 92-92. Retrieved July 14, 2009, from Academic Search Alumni Edition database.
SMITH, J. (2009, July). Fighting Physics: A Tough Battle. Communications of the ACM, 52(7), 60-65. Retrieved July 14, 2009, from Business Source Complete database.
