In-Memory Data Caching

The In-Memory technology is used to implement caching. Foresight Analytics Platform implements the following methods for working with data:

ROLAP. Get data by means of SQL queries from various DBMS including relational and columnar in real time.
In-Memory. Get required data from specialized multidimensional databases, special files or relational DBMS to RAM.

ROLAP

An SQL query to DBMS is executed on each new data address in real time. The execution result is sent to BI server and is converted to a data matrix that is stored in BI server RAM. The operation for handling SQL query execution result takes long time, especially if there is large amount of data obtained in the query execution result. The result is processed string by string, data reading speed is limited with the speed of a specific DBMS type. This approach is characterized as flexible and does not use much memory because one can load only data that is required at the moment.

In-Memory

In-Memory is a technology that ensures data loading from sources to RAM with later addressing to data on the go without having to address the hard drive or the DBMS, which speeds up data access and data processing significantly.

Data saved in the special format is loaded to RAM, conversion to matrix is not required because data is already stored in multidimensional format. One query is enough to get the entire matrix and place it in BI server RAM. Specialized multidimensional databases, special files or special data structures in relational DBMS can be used to store multidimensional data.

Data is cached in Foresight Analytics Platform using the In-Memory technology and global cache. For details about global cache see the Working with Global Cache article.

Data processing mechanisms in Foresight Analytics Platform:

Comparison of data processing mechanisms in Foresight Analytics Platform:

Criterion	ROLAP	Global cache
Scenarios of use on data processing	Universal use	Partial data reading and save.
Speedo of data cache creation (generation)	No cache	before 100 thous. data points in 1 sec.
Speed of loading data cache to RAM	No cache	before 1 mln. data points in 1 sec.
Data recovery after a crash	Not required. Data is stored in DBMS, their integrity is ensured by DBMS means.	On
Automatic translation of changed data in multidimensional cube with data in database tables	On	On
Additional expenses on saving data	No	No
Supported DBMS	All available for use in Foresight Analytics Platform.	All available for use in Foresight Analytics Platform. To save cache metadata, one should have a PostgreSQL, Postgres Pro, Oracle or MS SQL Server database.