Qlik Sense - Concurrent ReloadsArticle Number: 000033413 | Last Modified: 2018/06/12
- A reload task typically consumes at least one core, sometimes more
- With more concurrent tasks running than there are number of cores, the total time to complete a number of tasks that is larger than the number of cores does not improve with more parallel tasks running due to starvation of the server
- Each reload running is loaded into RAM. This means that the average RAM footprint grows with more parallel reloads
- With more parallel reloads than cores the probability for total starvation increases since the QIX engine is CPU hungry and other processes (for example, the operating system) might not get enough CPU cycles to perform their duties
- Use the guideline, NumberOfCPUCores - 2 in performing capacity planning on the possible amount of concurrent scheduled reloads.
- Scheduled reloads are queued, however, if not enough resources in the system
- To validate that environment needs proper capacity planning, go to C:\ProgramData\Qlik\Sense\Log\Scheduler\Trace\<Server>_System_Scheduler.txt, an example log shows below:
Domain\qvservice Engine connection released. 5 of 4 used
Domain\qvservice Engine connection 6 of 4 established
Domain\qvservice Request for engine-connection dequeued. Total in queue: 25
On a multi-node deployment, tasks will be balanced from the master node to any node(s) designated as slaves.
When there's a 3 nodes deployment (one master and two slaves) and each node is set to "Max concurrent reloads" = 4 and "Engine timeout" = 30:
- Master node receives a new task execution request.
- Master node checks the resource availability on each of the slave nodes.
- Master node assigns this task to the node which is least busy at the moment of triggering*.
*The balance check only involves CPU occupation and memory usage but NOT each Scheduler node queue for now. So even if the queue on a certain node has free space but the system usage is high (such as CPU 95% used, 2 tasks running), the new coming in task will be assigned to another fully queued node but with lower system usage (let's say CPU 60% used, but 4 tasks running)
Have a Question?
Search Qlik's Support Knowledge database or request assisted support for highly complex issues.Submit a case
Experiencing a serious issue, please contact us by phone. View phone numbers and hours by region.