Technical Approach

CloudDBAppliance will provide the three main capabilities for data processing: operational data management, analytics and big data in an energy-efficient high-performance hardware delivering a unique appliance to be deployed in cloud data centres to provide a reliable database as a service that can run business critical applications that today have not been migrated to the cloud due to the lack of predictable performance and trustworthiness that they need. The project will develop innovations across the following major areas:

New Efficient Hardware Enabling In-Memory Databases

The new Bullion hardware will provide a high modular server technology with blade and density-optimized servers with a many-core architecture that will provide over 800 cores and reach 140 TB of memory. This new hardware will be equipped with NVRAM (Non-Volatile Random Access Memory) as computer memory that enables a fully memory-driven data management. NVRAM will retain data even when electrical power is removed either from an unexpected power loss, system crash or from a normal system shutdown. This technology resides on the memory bus and runs at memory speed since it is DRAM-based.

The new Bullion will leverage the new 3D XPoint technology1. 3D XPoint is a form of non-volatile memory. 3D XPoint bridges the gap between DRAM and NAND. It is 1,000 times faster than NAND flash (the technology used by SSDs), but, still an order of magnitude slower than DRAM. Interestingly, it has a cost per bit that falls in between the two established technologies. The speed comes in part from the fact that 3D XPoint reads and writes data in very small sizes, similar to DRAM, but it is 10 times denser than DRAM. The new Bullion will save data movement that is very expensive today in cloud data centres, up to 50% of the energy consumption of a data centre. It will provide better MTBF (Medium Time Between Failures) thanks to hardware mechanisms. The new Bullion opens up a new paradigm for doing data management, but there are an innovative approach to exploit the large number of cores in a highly efficient manner by proposing a novel architecture based on applying the principles of distributed shared-nothing architectures within a large multi-core machine to avoid creating a large overhead due to sharing across many cores. LeanXcale will use core and memory affinity for the different subsystems to minimize the impact of context switching and the cost of accessing distant memory in the memory hierarchy. LeanXcale will build a new in-memory layer for interaction across subsystems based on efficient memory conflict-free abstractions that will avoid the cost and overhead of the TCP/IP-based communication that is too inefficient and costly to be used within the appliance.

Many-Core Fast In-Memory Analytical Engine

ActivePivot, the in-memory analytical engine developed by ActiveViam, will be re-architected for two purposes. First, to scale linearly to large number of cores. This is very challenging since internal thread synchronization becomes a bottleneck even with a few tens of cores. This will require a new architecture to be able to scale to over 100 cores. Second, to work efficiently in the many-core Bullion hardware. The goal is to be able to use the memory hierarchy of the three kinds of technologies RAM, 3DPoint, and NVMe (Non-Volatile Memory express) in an efficient way.

High Availability

In order to match the resilience of mainframes, different levels of high availability will be implemented. The first one will be a fault-tolerance mechanisms to detect process failures, JVM failures and failures of other subsystems to restart them automatically guaranteeing that the system continues to deliver service after restart. The second one will provide a high availability solution within the data centre, keeping a backup appliance that can take over the primary appliance when it fails. The third one will be a solution to provide high availability across data centres to be able to fail over at a different data centre in case of a disaster at data centre without losing any data.

Dynamic Migration

CloudDBAppliance will develop the capability of being able to migrate dynamically a database to another node in an online manner. In this way, when the resources for a particular node become insufficient for multiple tenants, one or more tenants can be migrated to another available node.

JVM with High Efficiency and Predictable Performance: Java technologies are good in avoiding problems due to memory leaks typical of technologies based on C/C++. However, java has an important weakness that prevents java from being used in high performance systems. One of the main obstacles is garbage collection, which results very intrusive and might stop the processing in the server for tens of seconds and even for several minutes with very large memories. This is something that time-critical applications or online transactional workloads cannot afford. In CloudDBAppliance the JVM will be enhanced with new garbage collection techniques to avoid the intrusive interruptions from the garbage collector. Thus, it will become possible to attain predictable performance using Java technologies, one of the main goals of the project.

The above listed innovations will be materialized in distinct subsystems, which will be integrated in the CloudDBAppliance platform.

as

On the top the different products are shown. On the bottom the two supporting technologies are shown: the Bullion hardware and the optimized JVM. The next layer is constituted by the features exhibited by the different products.