With the release of shared automations in Qlik Automate, it is now possible to run an automation in a Shared Space for users other than the owner. The automation will run using the owner’s Qlik account and any third-party connections as they were configured by the owner.

When an automation is run by another user, it is possible to retrieve this user’s information during the automation run. This article outlines how this is done.

This is only supported when the automation is run from the console or API. It is not supported for triggered automations when they are run from the trigger URL or webhook automations when they are run from the webhook event. The reason for this is that in these cases, no user subject is sent to the automation (triggered executions use the execution token instead of a user token, and webhook automations have no user involved when they are run by the webhook event).

However, if triggered or webhook automations are run manually or over API, it will be possible to retrieve the user's info.

Identifying the user who started the Automation

When an automation is run by any user, it is possible to retrieve the user id and user info for the user who executed the automation during the automation run.

1. Use the Get Automation Run block (A) from the Qlik Cloud Services connector and configure it with the Automation ID and Automation Run ID (C) formulas from the formula picker (B).
   
   
   
   
   
   
   
   
2. The Get Automation Run block will return the executedByID to identify who has started the automation:
   
   
   
   
3. Configure the Get User block to use the executedById parameter as input for the User Id parameter:
   
   
   
   
   
   
4. You can then use a Condition block to evaluate the returned value from the Get User block.
   
   For example, you can check if the user is a part of a predefined list of users:
   
   
   
   
   
   

Environment

• Qlik Cloud Automate
• Qlik Cloud

Where can I find the installed version or service release of my Qlik Sense Enterprise on a Windows server? 

You can locate the Qlik Sense Enterprise on Windows version in three locations:

• The Qlik Sense Hub
• The Qlik Sense Enterprise Management Console
• The Windows Installed Programs menu
• The Engine.exe on disk

The Qlik Sense Hub

1. Open the Qlik Sense Hub
2. Locate and click the Profile on the landing screen (corner on the right)
3. Click the About link 
   
   
   
   
4. The hub will display the installed version
   
   

The Qlik Sense Management Console

1. Open the Qlik Sense Management Console.
2. On the start screen, look in the bottom right corner for the version number.

The Windows Installed Programs menu 

1. Open the Windows Uninstall or change a program menu
2. Expand the window until the version is visible
   
   

The Engine.exe 

1. Open a Windows File explorer
2. Navigate to C:\Program Files\Qlik\Sense\Engine
3. Right-click the Engine.exe
4. Click Properties
5. Switch to the Details tab
6. The File version will be listed

Qlik Cloud Analytics does not list versions as the cloud version is always identical and current across the platform.

Unable to use data profiling to achieve Pattern Frequency Statistic on DB2. 

Resolution

As of R2025-06, the Pattern Frequency Statistic feature in Qlik Talend Studio > Data Profiling is not supported for DB2 databases. 

Workaround

Export the data in CSV format, define a file-delimited metadata type based on the CSV file, and then perform the profiling on that data.

For more information, see Pattern frequency statistics. 

An active Idea has been logged to support this feature for DB2. To vote on the idea, log in to Ideation before going to #495364 Talend Studio Data Profiling (DQ) support db2 Pattern Frequency Statistics indicators.

Environment

• Qlik Talend Studio

A Talend Snowflake Job is failing at the t<DB>Connection component when acquiring the connection to execute sql.  

The following error looks like below:

Getting the "Session no longer exists. New login required to access the service."

Resolution

Comparing both Timeout settings within Talend Studio and Snowflake

• In Studio -->Navigate to the snowflake connection component > Advanced Settings > Login Timeout 
• In Snowflake -->SHOW PARAMETERS LIKE '%timeout%';

Cause

The Snowflake or Talend Studio Snowflake component Login Timeout may have been set too low for what was required

Environment

• Talend Studio 
• Talend Data Integration 

If unable to access TAC using default credential (security@company.com/admin), you have forgotten your password.

You can reset it by following these steps:

1. Establish a connection to the TAC database.
2. In the TAC database, execute the following statement:

   UPDATE 'user' set 'password'=0x21232F297A57A5A743894A0E4A801FC3 where id =<userID>;​

   
   Note that `0x21...` corresponds to the encrypted password for "admin". To identify the <userID> associated with the user whose password you wish to update, execute this query: select id, login from user;
3. Using the new password, log in to Talend Administration Center once again.

Related Content 

Change the default password used to configure the TAC database

Environment

• Talend Administration Center 

The Talend Remote Engine Service fails to start, and systemd commands do not work as expected, resulting in the following error messages in the system logs:

Subject: Unit process exited

Defined-By: systemd

Support: https://access.redhat.com/support

An ExecStart= process belonging to unit talend-remote-engine.service has exited.

The process' exit code is 'exited' and its exit status is 126.

Jun 19 14:10:13 cmips-etl-ate-talend systemd[1]: talend-remote-engine.service: Failed with result 'exit-code'.

Subject: Unit failed

Defined-By: systemd

Support: https://access.redhat.com/support

The unit talend-remote-engine.service has entered the 'failed' state with result 'exit-code'.

Jun 19 14:10:13 cmips-etl-ate-talend systemd[1]: Failed to start karaf.

Subject: A start job for unit talend-remote-engine.service has failed

Defined-By: systemd

Support: https://access.redhat.com/support

A start job for unit talend-remote-engine.service has finished with a failure.

The job identifier is [Job Number] and the job result is failed.

Jun 19 14:17:50 cmips-etl-ate-talend systemd[1]: Starting karaf...

Subject: A start job for unit talend-remote-engine.service has begun execution

Defined-By: systemd

Support: https://access.redhat.com/support

A start job for unit talend-remote-engine.service has begun execution.

Resolution

Solution for A

1. Test Manual Startup
   To isolate whether the issue is with the service wrapper or the engine itself, try starting the engine manually:

   cd /path/to/Talend-RemoteEngine/bin

   If the engine starts successfully, the problem is with the systemd wrapper or permissions, not the engine itself.
2. Check and Correct File Permissions
   Ensure that the service wrapper scripts (e.g., trun.sh, talend-remote-engine-wrapper, or talend-remote-engine-service) are executable by the user running the service. For example:

   chmod +x /path/to/Talend-RemoteEngine/bin/trun.sh
   chmod +x /path/to/Talend-RemoteEngine/bin/talend-remote-engine-wrapper
   chmod +x /path/to/Talend-RemoteEngine/bin/talend-remote-engine-service

   Also, verify that the files are readable by the intended user and, if necessary, by the systemd process.

Solution for B

1. Address SELinux Restrictions
   By default, SELinux may block execution of the Talend Remote Engine service wrapper, resulting in permission errors even if file permissions are correct.
2. Temporarily disable SELinux
   To confirm if it is the source of the problem:

   sudo setenforce 0 

   Then, retry starting the service. If the service starts successfully, SELinux is the cause.
3. Permanently disable SELinux (if your security policy allows):
   Edit /etc/selinux/config and set:

   SELINUX=disabled

   Reboot the server for the change to take effect.
4. Alternatively, set SELinux to permissive mode:
   Edit /etc/selinux/config and set:

   SELINUX=permissive

   Reboot the server for the change to take effect.

    

Disabling SELinux reduces system security. Consider creating a custom SELinux policy to allow execution if you must keep SELinux enabled.

Cause

Exit status 126 in systemd indicates that the command was found but is not executable. This typically points to a permission issue on the script or binary that systemd is attempting to launch.

In this context of Talend Remote Engine, this is often caused by:

Cause A

Incorrect file permissions on the service wrapper scripts or binaries.

Cause B

SELinux policies prevents execution of the wrapper script or service file.

Environment

• Talend Cloud 
• Talend Studio 

The following release notes cover the Qlik PostgreSQL installer (QPI) version 1.2.0 to 2.0.0.

Content

• What's New
• 2.0.0 May 2025 Release Notes
• Known Limitations (2.0.0)
• 1.4.0 December 2023 Release Notes
• Known Limitations (1.4.0)
• 1.3.0 May 2023Release Notes
• Known Limitations (1.3.0)
• 1.2.0 Release Notes

What's New

• The PostgreSQL version used by QPI has been updated to 14.17
• An upgrade of an already upgraded external instance is now possible
• Support for an upgrade of the embedded 14.8 database was added
• Silent installs and upgrades are supported beginning with Qlik PostgreSQL Installer 1.4.0 and later. QPI can now be used with silent commands to install or upgrade the PostgreSQL database. (SHEND-973)

2.0.0 May 2025 Release Notes

Known Limitations (2.0.0)

• Rollback is not supported.
• The database size is not checked against free disk space before a backup is taken.
• Windows Server 2012 R2 does not support PostgreSQL 14.8. QPI cannot be used on Windows Server 2012 R2.
• QPI will only upgrade a PostgreSQL instance that only contains Qlik Sense Databases (QSR, SenseServices, QSMQ, Licenses, QLogs, etc.)

1.4.0 December 2023 Release Notes

Known Limitations (1.4.0)

• QB-24990: Cannot upgrade PostgreSQL if Qlik Sense was installed in a custom directory such as D:\Sense. See Qlik PostgreSQL Installer (QPI): No supported existing Qlik Sense PostgreSQL database found.
• Rollback is not supported.
• Database size is not checked against free disk space before a backup is taken.
• QPI can only upgrade bundled PostgreSQL database listening on the default port 4432. Using QPI to upgrade a standalone database or a database previously unbundled with QPI is not supported.
• Cannot migrate a 14.8 embedded DB to standalone
• Windows Server 2012R2 does not support PostgreSQL 14.8. QPI cannot be used on Windows Server 2012R2.

1.3.0 May 2023 Release Notes

A QlikView reload or Distribution task fails error message

System.IO.IOException: The process cannot access the file because it is being used by another process.

The task log will shows a successful reload but indicates the reloaded document can not be saved:

Information: Reload finished successfully
Information: Start saving document
Error: The sourcedocument failed to save.. Exception=System.IO.IOException: The process cannot access the file because it is being used by another process.
Error:    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
Error:    at System.IO.File.InternalReplace(String sourceFileName, String destinationFileName, String destinationBackupFileName, Boolean ignoreMetadataErrors)
Error:    at QVBWrapper.Document.ReplaceSourceFile(ILogBucket logBucket, String replaceFromFileName, String replaceToFilename)
Error:    at QVBWrapper.Document.Save(ILogBucket i_LogBucket, String i_SaveAsFileName)

File Access

After a successful reload of a QlikView Application (QVW) file, the old file is removed and replaced with a new file containing the new data. The error message indicates either that the old file could not be removed, or that file access is too slow to process the request. 

To identify if file access is too slow due to network latency, or due to disk read/write delays, review QlikView and its backend File Share System for details on the importance of fast file access, and how to monitor file shares.
In addition. 

In the case of file locks caused by processes locking the original file, the most common processes include:

• Anti-virus software
• Backup software
• QlikView services

The recommendation is to exclude QlikView folders from anti-virus scan. See What QlikView folder/files to exclude for anti-virus scanning for more details. 
Backup of QVW files must be planned and scheduled so that they do not interfere with QlikView tasks, to avoid file access conflicts.

The issue in general can only be resolved by identifying the process that is locking the file. See Process Monitoring - How to find locked files and the processes locking them for more details on how to identify the process that locks the QVW file.

Corrupted File

The file was previously corrupted.

1. Replace the QlikView document with a working copy
2. Rebuild the QlikView document from its project folder

You may be expericing some performance issues related to the task execution time or resuming file takes large storage problem. It is possible to reduce the resuming and recovery Logs in JobServer to solve them.

As the -Drecovery.enabled parameter is primarily used to control the creation and use of the recovery file, Setting it to false disables the feature and  meaning no resuming file will be generated or used. If you want to use recovery, you would leave this parameter enabled or set it to true (which is the default behavior)

This article briefly introuduces how to reduce job's resuming file if resuming file takes large storage

HOW TO

1. Stop Job Server.
2. Open the Job Server launch script start_rs.bat or start_rs.sh file and set the JVM arguments

   set MY_JVM_ARGS= -Drecovery.enabled=false    The default behavior is true

3. Restart Job Server.

Environment

• Talend Administration Center 
• Talend Data Integration 

In this article, we walk you through the requirements and process of how to upgrade and unbundle an existing Qlik Sense Repository Database (see supported scenarios) as well as how to install a brand new Repository based on PostgreSQL. We will use the Qlik PostgreSQL Installer (QPI).

For a manual method, see How to manually upgrade the bundled Qlik Sense PostgreSQL version to 12.5 version.

Using the Qlik Postgres Installer not only upgrades PostgreSQL; it also unbundles PostgreSQL from your Qlik Sense Enterprise on Windows install. This allows for direct control of your PostgreSQL instance and facilitates maintenance without a dependency on Qlik Sense. Further Database upgrades can then be performed independently and in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions. See How To Upgrade Standalone PostgreSQL.

Index

• Supported Scenarios
• Upgrades
• New installs
• Requirements
• Known limitations
• Installing anew Qlik Sense Repository Database using PostgreSQL
• Qlik PostgreSQL Installer - Download Link
• Upgrading an existing Qlik Sense Repository Database
• The Upgrade
• Next Steps and Compatibility with PostgreSQL installers
• How do I upgrade PostgreSQL from here on?
• Troubleshooting and FAQ
• Related Content

Video Walkthrough

Video chapters:

• 01:02 - Intro to PostgreSQL Repository
• 02:51 – Prerequisites
• 03:24 - What is the QPI tool?
• 05:09 - Using the QPI tool
• 09:27 - Removing the old Database Service
• 11:27 - Upgrading a stand-alone to the latest release
• 13:39 - How to roll-back to the previous version
• 14:46 - Troubleshooting upgrading a patched version
• 18:25 - Troubleshooting upgrade security error
• 21:15 - Additional config file settings

Supported Scenarios

Upgrades

The following versions have been tested and verified to work with QPI (1.4.0): 

Qlik Sense February 2022 to Qlik Sense November 2023.

If you are on a Qlik Sense version prior to these, upgrade to at least February 2022 before you begin.  

Qlik Sense November 2022 and later do not support 9.6, and a warning will be displayed during the upgrade. From Qlik Sense August 2023 a upgrade with a 9.6 database is blocked.

New installs

The Qlik PostgreSQL Installer supports installing a new standalone PostgreSQL database with the configurations required for connecting to a Qlik Sense server. This allows setting up a new environment or migrating an existing database to a separate host.

Requirements

• Review the QPI Release Notes before you continue

• Using the Qlik PostgreSQL Installer on a patched Qlik Sense version can lead to unexpected results. If you have a patch installed, either:

  • Uninstall all patches before using QPI (see Installing and Uninstalling Qlik Sense Patches) or
  • Upgrade to an IR release of Qlik Sense which supports QPI

• The PostgreSQL Installer can only upgrade bundled PostgreSQL database listening on the default port 4432.
• The user who runs the installer must be an administrator.
• The backup destination must have sufficient free disk space to dump the existing database  
• The backup destination must not be a network path or virtual storage folder. It is recommended the backup is stored on the main drive. 
• There will be downtime during this operation, please plan accordingly
• If upgrading to PostgreSQL 14 and later, the Windows OS must be at least Server 2016

Known limitations

• Cannot migrate a 14.8 embedded database to a standalone
• Using QPI to upgrade a standalone database or a database previously unbundled with QPI is not supported.
• The installer itself does not provide an automatic rollback feature.

Installing a new Qlik Sense Repository Database using PostgreSQL

1. Run the Qlik PostgreSQL Installer as an administrator 
   
   
2. Click on Install 
   
   
3. Accept the Qlik Customer Agreement 
   
   
4. Set your Local database settings and click Next. You will use these details to connect other nodes to the same cluster. 
   
   
5. Set your Database superuser password and click Next
   
   
6. Set the database installation folder, default: C:\Program Files\PostgreSQL\14

   Do not use the standard Qlik Sense folders, such as C:\Program Files\Qlik\Sense\Repository\PostgreSQL\ and C:\Programdata\Qlik\Sense\Repository\PostgreSQL\.

7. Set the database data folder, default: C:\Program Files\PostgreSQL\14\data

   Do not  use the standard Qlik Sense folders, such as C:\Program Files\Qlik\Sense\Repository\PostgreSQL\ and C:\Programdata\Qlik\Sense\Repository\PostgreSQL\.

8. Review your settings and click Install, then click Finish
   
   
9. Start installing Qlik Sense Enterprise Client Managed. Choose Join Cluster option.
   
   The Qlik PostgreSQL Installer has already seeded the databases for you and has created the users and permissions. No further configuration is needed.
   
   
10. The tool will display information on the actions being performed. Once installation is finished, you can close the installer.  
    
    If you are migrating your existing databases to a new host, please remember to reconfigure your nodes to connect to the correct host. How to configure Qlik Sense to use a dedicated PostgreSQL database 
    

Qlik PostgreSQL Installer - Download Link

Download the installer here.

Qlik PostgreSQL installer Release Notes

Upgrading an existing Qlik Sense Repository Database

The following versions have been tested and verified to work with QPI (1.4.0): 

February 2022 to November 2023.

If you are on any version prior to these, upgrade to at least February 2022 before you begin.  

Qlik Sense November 2022 and later do not support 9.6, and a warning will be displayed during the upgrade.  From Qlik Sense August 2023 a 9.6 update is blocked.

The Upgrade

1. Stop all services on rim nodes 
   
   
2. On your Central Node, stop all services except the Qlik Sense Repository Database 
   
   
   
   
3. Run the Qlik PostgreSQL Installer. An existing Database will be detected. 
   
   
   
   
4. Highlight the database and click Upgrade 
   
5. Read and confirm the (a) Installer Instructions as well as the Qlik Customer Agreement, then click (b) Next.
   
   
   
   
6. Provide your existing Database superuser password and click Next.
   
   
7. Define your Database backup path and click Next.
   
   
   
   
8. Define your Install Location (default is prefilled) and click Next.
   
   
   
   
9. Define your database data path (default is prefilled) and click Next.
   
   
    
10. Review all properties and click Upgrade.
    
    The review screen lists the settings which will be migrated. No manual changes are required post-upgrade.
    
    
    
    
11. The upgrade is completed. Click Close.
     
12. Open the Windows Services Console and locate the Qlik Sense Enterprise on Windows services.
    
    You will find that the Qlik Sense Repository Database service has been set to manual. Do not change the startup method. 
    
    
    
    You will also find a new postgresql-x64-14 service. Do not rename this service. 
    
    
    

     

13. Start all services except the Qlik Sense Repository Database service. 
    
    
14. Start all services on your rim nodes.
    
    
15. Validate that all services and nodes are operating as expected. The original database folder in C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\X.X_deprecated
    

     

16. Uninstall the old Qlik Sense Repository Database service.

    This step is required. Failing to remove the old service will lead the upgrade or patching issues.

    1. Open a Windows File Explorer and browse to C:\ProgramData\Package Cache
       
       
    2. From there, search for the appropriate msi file.
       
       
       
       If you were running 9.6 before the upgrade, search PostgreSQL.msi
       If you were running 12.5 before the upgrade, search PostgreSQL125.msi
       
       
    3. The msi will be revealed.
       
       
    4. Right-click the msi file and select uninstall from the menu.
       
       
       
       
    
    
17. Re-install the PostgreSQL binaries. This step is optional if Qlik Sense is immediately upgraded following the use of QPI. The Sense upgrade will install the correct binaries automatically. 
    

    Failing to reinstall the binaries will lead to errors when executing any number of service configuration scripts. 

    If you do not immediately upgrade:
    
    
    1. Open a Windows File Explorer and browse to C:\ProgramData\Package Cache
       
       
    2. From there, search for the .msi file appropriate for your currently installed Qlik Sense version
       
       
       
       For Qlik Sense August 2023 and later: PostgreSQL14.msi
       Qlik Sense February 2022 to May 2023: PostgreSQL125.msi
       
       
    3. Right-click the file
    4. Click Open file location
       
       
       
       
    5. Highlight the file path, right-click on the path, and click Copy
       
       
       
       
    6. Open a Windows Command prompt as administrator
       
       
    7. Navigate to the location of the folder you copied
       
       Example command line: 
       
       cd C:\ProgramData\Package Cache\{GUID}
       
       Where GUID is the value of the folder name.
       
       
    8. Run the following command depending on the version you have installed: 
       
       Qlik Sense August 2023 and later
       
       msiexec.exe /qb /i "PostgreSQL14.msi" SKIPINSTALLDBSERVICE="1" INSTALLDIR="C:\Program Files\Qlik\Sense"
       
       Qlik Sense February 2022 to May 2023
       
       msiexec.exe /qb /i "PostgreSQL125.msi" SKIPINSTALLDBSERVICE="1" INSTALLDIR="C:\Program Files\Qlik\Sense"
       
       This will re-install the binaries without installing a database. If you installed with a custom directory adjust the INSTALLDIR parameter accordingly. E.g. you installed in D:\Qlik\Sense then the parameter would be INSTALLDIR="D:\Qlik\Sense".
       
       
18. Finalize the process by updating the references to the PostgreSQL binaries paths in the SetupDatabase.ps1 and Configure-Service.ps1 files. For detailed steps, see Cannot change the qliksenserepository password for microservices of the service dispatcher: The system cannot find the file specified.

If the upgrade was unsuccessful and you are missing data in the Qlik Management Console or elsewhere, contact Qlik Support.  

Next Steps and Compatibility with PostgreSQL installers

Now that your PostgreSQL instance is no longer connected to the Qlik Sense Enterprise on Windows services, all future updates of PostgreSQL are performed independently of Qlik Sense. This allows you to act in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions.

Your PostgreSQL database is fully compatible with the official PostgreSQL installers from https://www.enterprisedb.com/downloads/postgres-postgresql-downloads. 

How do I upgrade PostgreSQL from here on?

See How To Upgrade Standalone PostgreSQL, which documents the upgrade procedure for either a minor version upgrade (example: 14.5 to 14.8) or a major version upgrade (example: 12 to 14). Further information on PostgreSQL upgrades or updates can be obtained from Postgre directly.

Troubleshooting and FAQ

1. If the installation crashes, the server reboots unexpectedly during this process, or there is a power outage, the new database may not be in a serviceable state. Installation/upgrade logs are available in the location of your temporary files, for example:
   
   C:\Users\Username\AppData\Local\Temp\2
   
   A backup of the original database contents is available in your chosen location, or by default in:
   
   C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\backup\X.X
   
   The original database data folder has been renamed to:
   
   C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\X.X_deprecated
   
   
2. Upgrading Qlik Sense after upgrading PostgreSQL with the QPI tool fails with:
   
   This version of Qlik Sense requires a 'SenseServices' database for multi cloud capabilities. Ensure that you have created a 'SenseService' database in your cluster before upgrading. For more information see Installing and configuring PostgreSQL.
   
   See Qlik Sense Upgrade fails with: This version of Qlik Sense requires a _ database for _.
   
   To resolve this, start the postgresql-x64-XX service.

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support. The video in this article was recorded in a earlier version of QPI, some screens might differ a little bit.

Related Content

Qlik PostgreSQL installer version 1.3.0 Release Notes  
Techspert Talks - Upgrading PostgreSQL Repository Troubleshooting
Backup and Restore Qlik Sense Enterprise documentation
Migrating Like a Boss
Optimizing Performance for Qlik Sense Enterprise
Qlik Sense Enterprise on Windows: How To Upgrade Standalone PostgreSQL
How-to reset forgotten PostgreSQL password in Qlik Sense
How to configure Qlik Sense to use a dedicated PostgreSQL database
Troubleshooting Qlik Sense Upgrades 

Qlik Sense allows for Data Encryption for Qlik Sense Apps (QVF) and Data (QVD) Files.

Qlik Sense Engine can encrypt data by using a data encryption key (DEK), which is generated from a certificate-based key encryption key (KEK). The DEK is unique to each encrypted file and is stored alongside the encrypted data (inside the QVF and QVD files) in an encrypted format using RSA. The industry standard AES-256 GCM is used as the data encryption algorithm. Decryption requires access to the same certificate as used during encryption, which mean that the KEK certificate thumbprint in the certificate store must match the thumbprint used for the DEK generation. 

Instructions

Data encryption can be enabled in the Service Cluster configuration of the Qlik Sense Management Console. See the Online Help at Qlik Sense Enterprise on Windows > Administer Qlik Sense Enterprise on Windows > Managing a Qlik Sense Enterprise on Windows site > QMC resources overview > Service cluster > Data encryption

Detailed Instructions with Example

This is a simple example of how to explore data encryption in Qlik Sense Enterprise on Windows.  

1. Create a new sample app, like the attached ascii-table.qvf
   1. Add a simple script to generate an ASCII table

      ASCII:
      Load
       if(RecNo()>=65 and RecNo()<=90,RecNo()-64) as Num,
       Chr(RecNo()) as AsciiAlpha,
       RecNo() as AsciiNum
      autogenerate 255
       Where (RecNo()>=32 and RecNo()<=126) or RecNo()>=160 ;

   2. Create a folder connection to a folder where the QVD can be written
   3. Generate QVD of a data table. Note, change the lib:// reference to match a valid data connection

      STORE ASCII INTO [lib://MyData (domain_administrator)/ascii.qvd] (QVD);

2. Reload the app
3. Copy the app and qvd files to allow comparison after enabling encryption 
   1. Default app location: C:\ProgramData\Qlik\Sense\Apps 
   2. The QVD location is per the app's folder data connection path
4. Create a certificate to test, following Encryption Certificates.
   1. Run Powershell as the Qlik Sense service account or log in to Windows as the Qlik Sense service account.
      
      

      Doing this with the correct account is crucial to make the generated certificate available to the service account.

      
      
      
   2. Generate a self-signed certificate to use as an encryption key
      
      Windows Server 2016
      
      

      New-SelfSignedCertificate -Subject "QlikSenseDataEncrytion" `
                                -KeyAlgorithm RSA `
                                -KeyLength 4096 `
                                -Provider "Microsoft Software Key Storage Provider" `
                                -KeyExportPolicy ExportableEncrypted `
                                -CertStoreLocation "cert:\CurrentUser\My"

      
      Windows Server 2012 R2
      
      

      New-SelfSignedCertificate -DnsName "QlikSenseDataEncrytion" `
                                -CertStoreLocation "cert:\CurrentUser\My"
      

   3. Validate that cert is available 
5. Get the cert thumbprint from generation result. This thumbprint can be used as key-encryption key (KEK) by Qlik Sense. 
   
   
   
   Note: When copying the certificate thumbprint, an invisible character may be added at the beginning of the certificate thumbprint. Verify the thumbprint before executing the command in PowerShell:
   
   
   
   
6. Enable encryption in Service Cluster settings per Data Encryption
   
   
   
   
7. Restart Qlik Sense Engine service
8. Reload the app to generate data encryption keys (DEK) for encrypted QVF and QVD files. 
   
   

Disable Encryption

Files remain encrypted after disabling encryption until the next following app reload or QVD generation. 

1. Disable encryption in QMC > Service Cluster
   1. Uncheck both encryption options
   2. Remove encryption key
      
2. Restart Qlik Sense Engine Service on all nodes
3. Complete a full successful app reload cycle, including QVD generators
   1. All QVF files have been decrypted
   2. All QVD files have been decrypted
      
      

File Comparison

Compare the unencrypted and encrypted files to validate successful encryption

Qlik Sense app (.qvf) file is a binary file, which makes it harder to visually confirm the encryption effect.

Encrypted app files have multiple references to ciphertext, which in turn refers to secrets used for the encryption.

App file without encryption has no such references. The cypher text portion of QVF represents the data encryption key (DEK) used to encrypt the app data and bookmarks. 

Encrypted data (.qvd) files has "Encryption Info" defined, which includes the data encryption (DEK) references.
Data part is also significantly different, even though both version have exactly the same static data content. 

Encryption Scope

• Encryption only applies at rest, meaning when stored on disk.  
• Data in memory is not encrypted
• Exported app (QVF) file is not encrypted
• Encrypted file (from apps storage folder) can not be imported in an other Qlik Sense instance.
  The import fails as the importing server is unable to parse the encrypted file. 
  
  

Related Content:

Qlik Sense on Windows: Data Encryption Key Rotation
Using Server Certificates for Data Encryption

This video will demonstrate how to install and configure Qlik-CLI for SaaS editions of Qlik Sense.

Content:

• Requirements
• Installation method One
• Installation Method Two
• Enable the Completion Feature (optional, but useful)
• Use Qlik-CLI
• Related Content
• Transcript

Requirements

• The user has access to a Qlik Cloud tenant
• The user has a professional license assigned in the tenant
• The user has the developer role assigned (required for API access)
• An API key can be obtained

Installation method One

1. Download Qlik-CLI from GitHub
2. Copy the executable to a local directory
3. Add the Qlik-CLI as an Environment Variable Path. 
   1. Open your Windows Control Panel
   2. Navigate to System 
   3. Open Advanced system settings in the leftmost menu
   4. Click Environment Variables 
   5. Locate Path in the User's System variables 
   6. Click Edit
   7. Click New
   8. Add the path to Qlik-CLI for example C:\Tools\Qlik-QLI\Modify the User Environment. Add path to Qlik-CLI. ie "C:\Tools\Qlik-QLI\"
   9. Confirm with OK until all windows are closed
4. Confirm the Qlik-CLI location path to verify it the installation completed successfully 
   1. Open PowerShell
   2. Execute:
      

      get-command qlik​

   3. Closer PowerShell

Installation Method Two

1. Download and install Chocolatey as documented on chocolatey.org.
2. Open PowerShell
3. Execute:
   

   choco install qlik-cli

4. This completes the installation.

Enable the Completion Feature (optional, but useful)

1. Verify if a PowerShell profile exists or create a new one
   1. Open PowerShell
   2. Execute:
      

      if ( -not (Test-Path $PROFILE) ) {
         echo "" > $PROFILE
      }​

2. Open the profile with the command notepad $PROFILE:
   

   qlik completion ps > "./qlik_completion.ps1" # Create a file containing the powershell completion.
   . ./qlik_completion.ps1                      # Source the completion.​

3. Save it. 
4. Restart PowerShell

Use Qlik-CLI

Advanced and additional instructions as seen in the video can be found at Qlik-CLI on Qlik.Dev. Begin with Get Started.

After adding a Global Rule to rename the target Schema (example: from PUBLIC to QLIK_TEST), the task may fail with an error indicating it was still trying to use the previous (example: PUBLIC) schema.

Reviewing the task JSON file to check the Global Rule definition shows no issues.

However, messages in the task logs indicate the table being created in the PUBLIC schema is not a task table. The table is an attrep change table:

00002308: 2024-03-25T16:51:38 [METADATA_MANAGE ]E: Failed (retcode -1) to execute statement: 'create or replace file format "PUBLIC"."ATTREP_FF_DEV_OS_STG_c1a25c9f_deef_f647_817b_db90dadf622b"

Resolution

Since the table mentioned in the error is not the actual target table, but a control table the resolution is to configure the control tables screen to also use the new schema (example: QLIK_TEST).

Screen Shot of the Control Tables Settings after setting schema to QLIK_TEST:

Related Content

Control Tables
Global Rules

When attempting to publish a standard DI job to Talend Management Console or an external artifact repository (regardless of if using Talend Studio or Talend CICD), you may be encountering the following error:

Caused by: org.talend.ipaas.services.inventory.InventoryException: workspace-service: failed to create artifact - Cannot publish artifact with the job type='data_service' because in one or more workspaces exists the same artifact with the different job type.

Other jobs may be able to publish to the same locations, even in the same project, without any specific issues.

Resolution

Two suggested solutions

• Put a tRESTClient (any ESB component) into the concurrent job, the “Build Type” combo box will be shown again, then select “Standalone” job, after that remove the tRESTClient, and save the job.
• Manually remove the BUILD_TYPE line from the job properties file.

  <additionalProperties xmi:id="_AVcxxO-mEe-Txxx_U4MkQ" key="BUILD_TYPE" value="OSGI"/>

Do not forget to re-publish your job at last

Cause

The issue, in many cases, may be an invalid entry within the job properties file, which still lists the job type as "Talend Runtime(OSGI)".
For this concurrent DI job, perhaps it was saved or exported at that time when there was a ESB component inside (Automatic Detective for build type). That’s why the job properties file had this line:

<additionalProperties xmi:id="_AVcxxO-mEe-Txxx_U4MkQ" key="BUILD_TYPE" value="OSGI"/>

During the publish-to-cloud via Talend studio or Talend CICD, the JobType will be set to “data_service” due to this wrongly set of BUILD_TYPE in the job properties file. With that invalid entry, the job cannot be used as a standalone job or microservice.

Environment

• Talend Studio 
• Talend Cloud 

Currently, in Qlik Application Automation it is not possible to export more than 100,000 cells using the Get Straight Table Data block.

Content:

• Full Automation
• Steps to set up the variables
• Automation structure

To overcome this limit, the workaround is to export records in batches from the Qlik Sense straight table to the cloud storage platform of your choice. The prerequisite is to have a unique numerical field in your dataset. If you don't have the unique field in your dataset, you can add it using RowNo() function in the load script as shown below. This counts the rows in the dataset.

In this example, we will export data from the Qlik Sense straight table to Dropbox as a CSV file.

You can also find an exported version of this automation and application attached to this article. More information on importing automation can be found here.

Full Automation

Automation Part 1

Automation Part 2

Automation Part 3

Steps to set up the variables

1. vAppId: Variable used to define your app ID.
2. vSheetId: Variable used to define your sheet ID.
3. vTableId: Variable used to define your straight table ID.
4. vFieldName: Variable used to define the field name on which the straight table data should be filtered based on the selection applied using the Select Field Value By Query block.

Automation structure

1. Add the Create Measure block to the automation and configure the Measure Expression parameter by providing an expression that calculates the number of rows in the dataset. In this example, I have added RowNo( ) as a unique field to the dataset, so the expression used is count([RowNo( )]).
2. Add the Get Measure block to get the number of rows in the dataset.
3. Store the number of rows obtained from the previous step in the vNRecords variable.
4. Delete the measure created using the Delete Measure block.
5. Add the vNLoops variable to calculate the number of loops that can be used to export straight table data in batches. Please note the formula used in the variable. Edit it to suit your needs. Change the 10000 value as you want but keep in mind that too many records could exceed the Get Straight Table Data block's 100,000 cells limit.
   
   
   
   
6. The Custom Code block creates a list with items 1 to n where n is the number of loops calculated in the previous step.
7. Add Create File on Dropbox block to create a new CSV file in Dropbox. You can select any other cloud storage connector, just remember to configure the relevant value in the Path parameter. The Qlik Sense straight table data will be exported to this file.
8. Add the Select Field Values By Query block to apply selection to the unique identifier created in the dataset to fetch only one record from the Get Straight Table Data block.
9. Add Write Line To File on Dropbox block to add straight table keys as column headers to the CSV file.
10. Add Loop block to loop over the list created using the Custom Code block.
11. Add Condition block to check if it's iterating over the 1st item, if yes set vRecordsStart variable value to 1, else set vRecordsStart variable value to vRecordsEnd variable. The vRecordsEnd variable represents the last record that needs to be fetched from the straight table during each iteration. Please note the formula used in the variable. The value under the Value 1 parameter in the Multiply formula should match the value provided in the Divide formula of the vNLoops variable.
    
    
    
    
12. Add the Select Field Values By Query block to apply selection to the unique identifier created in the dataset to fetch the records between vRecordsStart and vRecordsEnd variables from the Get Straight Table Data block.
13. Add Write Line To File on Dropbox block within the Get Straight Table Data block which writes each straight table data into the CSV file. Configure it to use the output from the Get Straight Table Data block as the Data parameter.
14. Add Save And Close File on Dropbox block and configure the file parameter to use the CSV file we have created.

Environment

• Qlik Cloud
• Qlik Automate 

An OAuth client has admin.apps, admin.automations and admin.spaces scopes.

When using it with API calls to run automations in a shared space, the result is 403 insufficient permissions.

The assumption is that the admin scopes should be enough to execute the automation.

Resolution

Explicitly add the OAuth client as a user on the tenant and give the user Can manage permissions. 

Cause

It is the same concept as a Tenant Admin.

A Tenant Admin can see everything, but to open apps in a space or reload them, the admin user must be explicitly added to the space.

The same concept applies to the OAuth admin scopes.

Environment

• Qlik Automate
• Qlik Cloud

This article documents how to create an AWS Amazon DynamoDB table using Qlik Automate. The example automation is attached to this article and can be imported. See How to import and export automations for details.

1. Choose the Amazon DynamoDB Create Table block
   
   
   
   The Key Schema and Attribute Definitions input field cannot simply have a JSON structure applied (such as "[{'AttributeName': 'ID', 'KeyType': 'HASH'}]"). The key is to create variables and lists and apply these lists as inputs.
   
   
   • Key Schema example:
     
     First, create an AttributeName variable with the Key AttributeName and the Value ID
     
     
     
     
   • Repeat this for the KeyType variable with the Key KeyType and Value HASH.
     
     
     
     
2. Create a Variable List with the Variable keyList (list) and Item Key
   
   
   
   
3. Apply the Variable List to the Key Schema field:
   
   

The finished automation:

Notes:

• If the billing is PROVISIONED, the ProvisionedThroughput must explicitly include both ReadCapacityUnits and WriteCapacityUnits:
  

  "ProvisionedThroughput": {
    "ReadCapacityUnits": 5,
    "WriteCapacityUnits": 5}

• There is a known issue when using PAY_PER_REQUEST as the billing mode. This should be fixed in the next releases.

Environment

• Qlik Cloud Analytics
• Qlik Automate

The Qlik Sense Desktop or Server installation fails with:

INSTALLATION FAILED
AN ERROR HAS OCCURRED
For detailed information see the log file.
 

The installation logs (How to read the installation logs for Qlik products) will read:

Error 0x80070643: Failed to install MSI package.
Error 0x80070643: Failed to configure per-user MSI package.
Detected failing msi: DemoApps

Error 0x80070643: Failed to install MSI package.
Error 0x80070643: Failed to configure per-user MSI package.
Detected failing msi: SenseDesktop
Applied execute package: SenseDesktop, result: 0x80070643, restart: None
Error 0x80070643: Failed to execute MSI package.
ProgressTypeInstallation
Starting rollback execution of SenseDesktop

CAQuietExec:  Entering CAQuietExec in C:\WINDOWS\Installer\MSIE865.tmp, version 3.10.2103.0
CAQuietExec:  "powershell" -NoLogo -NonInteractive -InputFormat None 
CAQuietExec:  Error 0x80070002: Command failed to execute.
CAQuietExec:  Error 0x80070002: QuietExec Failed
CAQuietExec:  Error 0x80070002: Failed in ExecCommon method
CustomAction CA_ConvertToUTF8 returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)
MSI (s) (74:54) [10:44:37:941]: Note: 1: 2265 2:  3: -2147287035 
MSI (s) (74:54) [10:44:37:942]: User policy value 'DisableRollback' is 0
MSI (s) (74:54) [10:44:37:942]: Machine policy value 'DisableRollback' is 0
Action ended 10:44:37: InstallFinalize. Return value 3.

The Failed to Install MSI Package can have a number of different root causes.

Install C++ redistributable

Dependencies may be missing. Install the latest C++ redistributable. 

.NET Framework update installation error: "0x80070643" or "0x643".

This issue may occur if the MSI software update registration has become corrupted, or if the .NET Framework installation on the computer has become corrupted (source: Microsoft, KB976982). 

Repair or reinstall .NET framework. 

PowerShell path not set as a system variable

How to troubleshoot:

Option 1

1. Start "Advanced System Settings" ->  "Advanced" tab -> "Environment Variables"
2. Select "Path" from "System variables" and Click Edit
3. Check if the PowerShell path is listed
4. If it is missing, add "%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\" and save the change
   
   Please make sure powershell.exe can be evoked successfully from %SYSTEMROOT%\System32\WindowsPowerShell\v1.0\ before adding the path.
   
   

1. Run the batch file in the article Windows Commands To Retrieve Environment Information v2.14
2. Open the result file and find the "Environment Variables" part
   

   ********************************************        
   *********  Environment Variables  **********        
   ********************************************    
   .......
   
   Path=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Users\administrator\AppData\Local\Microsoft\WindowsApps;C:\WINDOWS\system32\inetsrv\​

3. If the path of PowerShell is missing, refer to option 1 to add into the system variables. 

Web Platform Installer Extensions required

Install the Web Platform Installer Extension that includes the latest components of the Microsoft Web Platform, including Internet Information Services (IIS), SQL Server Express, .NET Framework and Visual Studio.

More information about the tool on the Microsoft page. 

Windows Updates are pending

Verify if there are pending Windows updates. Complete them and try again.

The Installer is not hosted on a local drive

The installation may fail if the installer is being executed from a network drive. Copy the installer to your local drive.

Environment:

Qlik Sense Desktop 

When using a route that has been migrated from an earlier version of Talend 8 to a newer patch (R2025-02 or a later version), it is possible that certain routes may not migrate correctly. If a cREST component exists within the route and incorporates camel configuration flags within one or multiple cConfig components, the Job may encounter difficulties in successfully passing the header after building and running in the newer Studio. In the log of the route/service, subsequent to the headers being passed, the termination of the route/service (upon failure) may resemble the following:

BodyType: null
Body: [Body is null]

Cause

Starting with the R2025-02 Monthly Patch, Qlik move to Camel 4 for Routes/Services may result in existing flags being set differently within the code. Furthermore, certain flags that were compatible with Camel 2 may no longer function in Camel 4 or may have undergone changes.

Resolution

If the flags "camelContext.setStreamCaching(true)", "jakarta.ws.rs.core.Response", and/or any Camel-specific flags were incorporated into the route/service prior to its migration to patch R2025-02 (or a later version), kindly remove or modify these flags from the route.

Environment

• Talend Data Integration 8.0.1