How Much Memory Is Required For A Qlik Server Deployment?

ToniKautto · Jan 19, 2023 7:52:49 AM

The Qlik Indexing (QIX) Engine allocates and utilizes memory during data reloads and calculation of visualizations. The QIX engine and memory allocation works the same way in Qlik Sense Enterprise, Qlik Analytics Platform, QlikView Server and QlikView Publisher.

Index:

Memory Utilization During Data Reload
Memory Utilization During User Activity
Memory Allocation On Hardware
Linear Scaling Of Resources

For system requirements, see: System requirements for Qlik Sense Enterprise | System Requirements for QlikView Server

Note that system requirements for basic installation do not equate to performance. Each Qlik system will have a different memory, CPU, and disk requirement depending on data, apps, and user behaviour.

If you are looking for information on how to scale your environment, proceed with this article. For direct assistance, engage with our Professional Services, who can provide you with guidance based on your requirements.

Memory Utilization During Data Reload

Memory is allocated during reload to hold loaded data records and cache results of data value processing. The amount of memory required for the loaded data depends on the nature of the data, in terms of number or rows and symbols required to represent the data. See How Is Data Stored In Qlik Sense And QlikView? for more details on how data is stored in memory. If the loaded data is manipulated during load, this will lead to additional memory allocation to hold cached results of the aggregation or manipulation. After finished reload the cached values related to the reload are purged from memory.

In server side reload the reloaded data model is stored from memory to file, followed by the data in memory being purged. After this there is no remaining memory allocation related to the reload. The application will be loaded into memory again once opened by the server or reloaded again.
In desktop client the reloaded data model remains in memory as long as the application remains open in the client. If the data model size increased during reload, the memory allocation can appear larger after reload compared to before reload.

Any manipulation or processing of data values require memory in order to store and handle the associated calculation. As a consequence a reload can consume more memory than the actual loaded data size, if the loaded data is manipulated during load. After finished reload the cached values related to the reload are purged from memory.

The best way to evaluate and estimate the memory required for reload, is to simple reload an app and observe the memory allocation. There can be room for optimisation of reload structure, if memory utilisation is considered too large. It is important to consider data growth over time, when estimating memory need.

Memory Utilization During User Activity

Memory allocation during user activity and interaction consists of three parts;

Loading the app into memory	The app size in memory is static and reflects data model and content of the app file. The app size on disk will not directly reflect the size allocated in memory, as the file is partly uncompressed during opening and loading into memory.
Ongoing user session data	The session data per user is expected to require a slim slice of memory. The allocation size primarily is reflected by the size of the data model and number of (alternate) states in the app. The total session size will multiply with the number of active user sessions.
Cached calculation results	Cached calculations results are used to reuse previous processing, and thereby save time and give users a high performance on selection changes and visualization updates.

The parts above lead to a memory usage history as described in Technical Brief: QlikView Server Memory Management And CPU Utilization and exemplified in below images where one or multiple open and accessed apps.

User activity memory scaling 01.png

User activity memory scaling 02.png

Estimate the required memory to open a specific app by opening it on a blank sheet. There are no calculations without visualizations, so this will indicate the minimum required memory to open the app. By opening all apps that are expected to be accessed at the same time, it is possible to estimate the total minimum required memory for opening all the relevant apps.

By opening the apps with a user, the above estimation of memory for opening an app includes one user session. Accessing the apps with for example 10 users enable estimating the memory required for the sessions per user. Multiplying the estimated user session size with the number of anticipated simultaneous users gives an estimation of required memory for expected concurrent user sessions.

The above estimations added on top of an estimation of how much memory is required for the OS and other processes, will tell how much memory is left for the Qlik server cache. A larger cache enables more reuse of previous calculations, and enables better performance and user experience. Note, reload of the app can lead to cache being purged as previous results can not longer be assumed as valid.

The memory consumption in QIX engine is considered linear, so scaling needs can be quite easy to evaluate once a base line of expectation has been estimated.

Memory Allocation On Hardware

Higher amounts of RAM will allow for more cached result sets and typically better performance. Supported memory configurations for best performance should be confirmed with the main board manufacturer. The manufacturer should have guidelines for which configurations of RAM can be deployed whilst keeping the bus speed as high as possible. The QIX engine allocates memory uniformly over all available RAM, meaning that it is important to always have the amount of memory per CPU socket evenly distributed

Linear Scaling Of Resources

Qlik Scalability Center has tested and evaluated how memory scales over different sizes of apps and user loads. These test results show that by adding up the resource usage of individual Qlik apps, it is possible to get a close approximation from a resources perspective of what resources are needed when the apps run in parallel.

How Much Memory Is Required For A Qlik Server Deployment?