When replicating to Oracle target, in some cases you may get ORA-12899 error.

This error may happen in a situation where your Oracle target has NLS_LENGTH_SEMANTICS set to ‘byte’ and the column being replicated from the source endpoint to the Oracle target endpoint includes at Unicode character/s that is presented by more than one byte. In this case, for example, when you replicate a varchar (10), it will be created as varchar (10 byte). Therefore, if the replicated data in the source endpoint includes a Unicode character that in UTF8 is presented by more than one byte, it could be that although the source is only 10 characters, on target endpoint (where replicate works in utf8), it would require more than 10 bytes for presenting its value. This will result with ORA-12899 error.

In general, when working with Oracle, the NLS_LENGTH_SEMANTICS setting determines how the CHAR and VARCHAR2 columns will be created. i.e., it enables you to create CHAR and VARCHAR2 columns using either byte or character length semantics.

Environment

• Replicate with Oracle target endpoint

Resolution

1. Stop the task
2. Open the Oracle target endpoint, go to under Advanced tab-->Internal Parameters.
   In the search box type charLengthSemantics and set it to CHAR. This will cause Replicate to create CHAR and VARCHAR2 columns using char semantics. For example: COL1 VARCHAR2(10 CHAR) instead of COL1 VARCHAR2(10 BYTES), and this will make sure that column length will be long enough to hold its value (including Unicode characters) and thus eliminate the ORA-12899 error.

Note:

• This internal property in Replicate charLengthSemantics determines whether to use the oracle default of NLS_LENGTH_SEMANTICS or override it.
• The NLS_LENGTH_SEMANTICS/charLengthSemantics setting affects any CHAR and VARCHAR2 columns in any table that Replicate creates in the target endpoint

With February 2018 of Qlik Sense, it is possible to capture granular usage metrics from the Qlik in-memory engine based on configurable thresholds.  This provides the ability to capture CPU and RAM utilization of individual chart objects, CPU and RAM utilization of reload tasks, and more.

Also see Telemetry logging for Qlik Sense Administrators in Qlik's Help site.

Click here for Video Transcript

Enable Telemetry Logging

In the Qlik Sense Management Console, navigate to Engines > choose an engine > Logging > QIX Performance log level.  Choose a value:

• Off: No logging will occur
• Error: Activity meeting the ‘error’ threshold will be logged
• Warning: Activity meeting the ‘error’ and ‘warning’ thresholds will be logged
• Info: All activity will be logged

• Repeat for each engine for which telemetry should be enabled.

Set Threshold Parameters

• Edit C:\ProgramData\Qlik\Sense\Engine\Settings.ini   If the file does not exist, create it.  You may need to open the file as an administrator to make changes.
• Set the values below.  It is recommended to start these threshold values and increase or decrease them as you become more aware of how your particular environment performs.  Too low of values will create very large log files.

 
[Settings 7]
ErrorPeakMemory=2147483648
WarningPeakMemory=1073741824
ErrorProcessTimeMs=60000
WarningProcessTimeMs=30000
 

• Save and close the file. 
• Restart the Qlik Sense Engine Service Windows service.
• Repeat for each engine for which telemetry should be enabled.

 
Parameter Descriptions
 

• ErrorPeakMemory: Default 2147483648 bytes (2 Gb).  If an engine operation requires more than this value of Peak Memory, a record is logged with log level ‘error’.  Peak Memory is the maximum, transient amount of RAM an operation uses. 
• WarningPeakMemory: Default 1073741824 bytes (1 Gb).  If an engine operation requires more than this value of Peak Memory, a record is logged with log level ‘warning’.  Peak Memory is the maximum, transient amount of RAM an operation uses. 
• ErrorProcessTimeMs: Default 60000 millisecond (60 seconds).  If an engine operation requires more than this value of process time, a record is logged with log level ‘error’.  Process Time is the end-to-end clock time of a request.
• WarningProcessTimeMs: Default 30000 millisecond (30 seconds).  If an engine operation requires more than this value of process time, a record is logged with log level ‘warning’.  Process Time is the end-to-end clock time of a request.

 
Note that it is possible to track only process time or peak memory.  It is not required to track both metrics.  However, if you set ErrorPeakMemory, you must set WarningPeakMemory.  If you set ErrorProcessTimeMs, you must set WarningProcessTimeMs.

Reading the logs

• To familiarize yourself with logging basics, please see help.qlik.com

Note: Currently telemetry is only written to log files.  It does not yet leverage the centralized logging to database capabilities.
 

• Telemetry data is logged to C:\ProgramData\Qlik\Sense\Log\Engine\Trace\<hostname>_QixPerformance_Engine.txt
• and rolls to the ArchiveLog folder in your ServiceCluster share.
• In addition to the common fields found described), fields relevent to telemetry are:
  • Level: The logging level threshold the engine operation met.
  • ActiveUserId: The User ID of the user performing the operation.
  • Method: The engine operation itself.  See Important Engine Operations below for more. 
  • DocId: The ID of the Qlik application.
  • ObjectId: For chart objects, the Object ID of chart object.
  • PeakRAM: The maximum RAM an engine operation used.
  • NetRAM: The net RAM an engine operation used.  For hypercubes that support a chart object, the Net RAM is often lower than Peak RAM as temporary RAM can be used to perform set analysis, intermediate aggregations, and other calculations.
  • ProcessTime: The end-to-end clock time for a request including internal engine operations to return the result.
  • WorkTime: Effectively the same as ProcessTime excluding internal engine operations to return the result.  Will report very slightly shorter time than ProcessTime. 
  • TraverseTime: Time spent running the inference engine (i.e, the green, white, and grey).

 
Note: for more info on the common fields found in the logs please see help.qlik.com.

Important Engine Operations

The Method column details each engine operation and are too numerous to completely detail.  The most relevent methods to investigate are as follows and will be the most common methods that show up in the logs if a Warning or Error log entry is written. 
 

Comments

• For best overall representation of the time it takes for an operation to complete, use ProcessTime.
• About ERROR and WARNING log level designations: These designations were used because it conveniently fit into the existing logging and QMC frameworks.  A row of telemetry information written out as an error or warning does not at all mean the engine had a warning or error condition that should require investigation or remedy unless you are interested in optimizing performance.  It is simply a means of reporting on the thresholds set within the engine settings.ini file and it provides a means to log relevant information without generating overly verbose log files.

Qlik Telemetry Dashboard:

Once the logs mentioned above are created, the Telemetry Dashboard for Qlik Sense can be downloaded and installed to read the log files and analyze the information.
The Telemetry Dashboard provides the ability to capture CPU and RAM utilization of individual chart objects, CPU and RAM utilization of reload tasks, and more.

For additional information including installation and demo videos see Telemetry Dashboard - Admin Playbook 
The dashboard installer can be downloaded at:  https://github.com/eapowertools/qs-telemetry-dashboard/wiki.  NOTE: The dashboard itself is not supported by Qlik Support directly, see the Frequently Asked Questions about the dashboard,  to report issues with the dashboard, do so on the "issues" webpage.

1. Right-click installer file and use "Run as Administrator". The files will be installed at C:\Program Files\Qlik\Sense
2. Once installed, you will see 2 new tasks, 2 data connections and 1 new app in the QMC.
3. In QMC Change the ownership of the application to yourself, or the user you want to open the app with.
4. Click on the 'Tasks' section in the QMC, click once on 'TelemetryDashboard-1-Generate-Metadata', then click 'Start' at the bottom. This task will run, and automatically reload the app upon completion.
5. Use the application from the hub to browse the information  by sheets

Sometimes we need to store column's before-image data in the target table. This is useful  if we want to store both of the before-image and after-image of the columns values in the target table for downstream apps usage. 

Environment

• Qlik Replicate All supported versions

Detailed Steps:

Under Apply Changes mode (Store changes mode is turn off),  in the table setting by adding a new column in transformation (name it as "prevenient_salary" in this sample), the variable expression is like $BI__<columnName> where $BI__ is a mandatory prefix (which instructs Replicate to capture the before-image data) and <columnName> is the original table column. For example if the original table column name is SALARY then $BI__SALARY is the column before-image data:

If the column SALARY value is updated from 22 to 33 in source side, then before the UPDATE the target table row looks like:

after the UPDATE is applied to target table the row looks like:

In this sample the before-image value is 22, the after-image value is 33.

Related Content

before-image data can be used in filer also, see sample here .

Qlik Replicate 

Scenario:

1. Have Qlik-Cli installed and configured (see How to install Qlik-CLI for Qlik Sense on Windows) for more insight on how to install and configure Qlik-Cli)
2. Create a PowerShell script using the following code as a base:

! Note that the code is provided as is and will need to be modified to suit specific needs. For assistance, please contact the original creators of Qlik-Cli.

Connect-Qlik -computername "QlikSenseServer.company.com"
# This will export the custom security rules and output them as secrules.json
Get-QlikRule -filter "type eq 'Custom' and category eq 'security'" -full -raw  | ConvertTo-Json  | Out-File secrules.json
# This will import secrules.json into a Sense site
# The ideal use case would be to move the developedsecurity rules from one environment to another, so connecting to another Sense site would likely be needed here
Get-Content -raw .\secrules.json |% {$_ -replace '.*"id":.*','' } | ConvertFrom-Json | Import-QlikObject

Environment:

Qlik Sense Enterprise on Windows , all versions

Where can I find the installed version or service release of my Qlik Sense Enterprise on a Windows server? 

You can locate the Qlik Sense Enterprise on Windows version in three locations:

• The Qlik Sense Hub
• The Qlik Sense Enterprise Management Console
• The Windows Installed Programs menu
• The Engine.exe on disk

The Qlik Sense Hub

1. Open the Qlik Sense Hub
2. Locate and click the Profile on the landing screen (corner on the right)
3. Click the About link 
   
   
   
   
4. The hub will display the installed version
   
   

The Qlik Sense Management Console

1. Open the Qlik Sense Management Console.
2. On the start screen, look in the bottom right corner for the version number.

The Windows Installed Programs menu 

1. Open the Windows Uninstall or change a program menu
2. Expand the window until the version is visible
   
   

The Engine.exe 

1. Open a Windows File explorer
2. Navigate to C:\Program Files\Qlik\Sense\Engine
3. Right-click the Engine.exe
4. Click Properties
5. Switch to the Details tab
6. The File version will be listed

Qlik Cloud Analytics does not list versions as the cloud version is always identical and current across the platform.

With the release of shared automations in Qlik Automate, it is now possible to run an automation in a Shared Space for users other than the owner. The automation will run using the owner’s Qlik account and any third-party connections as they were configured by the owner.

When an automation is run by another user, it is possible to retrieve this user’s information during the automation run. This article outlines how this is done.

This is only supported when the automation is run from the console or API. It is not supported for triggered automations when they are run from the trigger URL or webhook automations when they are run from the webhook event. The reason for this is that in these cases, no user subject is sent to the automation (triggered executions use the execution token instead of a user token, and webhook automations have no user involved when they are run by the webhook event).

However, if triggered or webhook automations are run manually or over API, it will be possible to retrieve the user's info.

Identifying the user who started the Automation

When an automation is run by any user, it is possible to retrieve the user id and user info for the user who executed the automation during the automation run.

1. Use the Get Automation Run block (A) from the Qlik Cloud Services connector and configure it with the Automation ID and Automation Run ID (C) formulas from the formula picker (B).
   
   
   
   
   
   
   
   
2. The Get Automation Run block will return the executedByID to identify who has started the automation:
   
   
   
   
3. Configure the Get User block to use the executedById parameter as input for the User Id parameter:
   
   
   
   
   
   
4. You can then use a Condition block to evaluate the returned value from the Get User block.
   
   For example, you can check if the user is a part of a predefined list of users:
   
   
   
   
   
   

Environment

• Qlik Cloud Automate
• Qlik Cloud

Unable to use data profiling to achieve Pattern Frequency Statistic on DB2. 

Resolution

As of R2025-06, the Pattern Frequency Statistic feature in Qlik Talend Studio > Data Profiling is not supported for DB2 databases. 

Workaround

Export the data in CSV format, define a file-delimited metadata type based on the CSV file, and then perform the profiling on that data.

For more information, see Pattern frequency statistics. 

An active Idea has been logged to support this feature for DB2. To vote on the idea, log in to Ideation before going to #495364 Talend Studio Data Profiling (DQ) support db2 Pattern Frequency Statistics indicators.

Environment

• Qlik Talend Studio

A Talend Snowflake Job is failing at the t<DB>Connection component when acquiring the connection to execute sql.  

The following error looks like below:

Getting the "Session no longer exists. New login required to access the service."

Resolution

Comparing both Timeout settings within Talend Studio and Snowflake

• In Studio -->Navigate to the snowflake connection component > Advanced Settings > Login Timeout 
• In Snowflake -->SHOW PARAMETERS LIKE '%timeout%';

Cause

The Snowflake or Talend Studio Snowflake component Login Timeout may have been set too low for what was required

Environment

• Talend Studio 
• Talend Data Integration 

If unable to access TAC using default credential (security@company.com/admin), you have forgotten your password.

You can reset it by following these steps:

1. Establish a connection to the TAC database.
2. In the TAC database, execute the following statement:

   UPDATE 'user' set 'password'=0x21232F297A57A5A743894A0E4A801FC3 where id =<userID>;​

   
   Note that `0x21...` corresponds to the encrypted password for "admin". To identify the <userID> associated with the user whose password you wish to update, execute this query: select id, login from user;
3. Using the new password, log in to Talend Administration Center once again.

Related Content 

Change the default password used to configure the TAC database

Environment

• Talend Administration Center 

The Talend Remote Engine Service fails to start, and systemd commands do not work as expected, resulting in the following error messages in the system logs:

Subject: Unit process exited

Defined-By: systemd

Support: https://access.redhat.com/support

An ExecStart= process belonging to unit talend-remote-engine.service has exited.

The process' exit code is 'exited' and its exit status is 126.

Jun 19 14:10:13 cmips-etl-ate-talend systemd[1]: talend-remote-engine.service: Failed with result 'exit-code'.

Subject: Unit failed

Defined-By: systemd

Support: https://access.redhat.com/support

The unit talend-remote-engine.service has entered the 'failed' state with result 'exit-code'.

Jun 19 14:10:13 cmips-etl-ate-talend systemd[1]: Failed to start karaf.

Subject: A start job for unit talend-remote-engine.service has failed

Defined-By: systemd

Support: https://access.redhat.com/support

A start job for unit talend-remote-engine.service has finished with a failure.

The job identifier is [Job Number] and the job result is failed.

Jun 19 14:17:50 cmips-etl-ate-talend systemd[1]: Starting karaf...

Subject: A start job for unit talend-remote-engine.service has begun execution

Defined-By: systemd

Support: https://access.redhat.com/support

A start job for unit talend-remote-engine.service has begun execution.

Resolution

Solution for A

1. Test Manual Startup
   To isolate whether the issue is with the service wrapper or the engine itself, try starting the engine manually:

   cd /path/to/Talend-RemoteEngine/bin

   If the engine starts successfully, the problem is with the systemd wrapper or permissions, not the engine itself.
2. Check and Correct File Permissions
   Ensure that the service wrapper scripts (e.g., trun.sh, talend-remote-engine-wrapper, or talend-remote-engine-service) are executable by the user running the service. For example:

   chmod +x /path/to/Talend-RemoteEngine/bin/trun.sh
   chmod +x /path/to/Talend-RemoteEngine/bin/talend-remote-engine-wrapper
   chmod +x /path/to/Talend-RemoteEngine/bin/talend-remote-engine-service

   Also, verify that the files are readable by the intended user and, if necessary, by the systemd process.

Solution for B

1. Address SELinux Restrictions
   By default, SELinux may block execution of the Talend Remote Engine service wrapper, resulting in permission errors even if file permissions are correct.
2. Temporarily disable SELinux
   To confirm if it is the source of the problem:

   sudo setenforce 0 

   Then, retry starting the service. If the service starts successfully, SELinux is the cause.
3. Permanently disable SELinux (if your security policy allows):
   Edit /etc/selinux/config and set:

   SELINUX=disabled

   Reboot the server for the change to take effect.
4. Alternatively, set SELinux to permissive mode:
   Edit /etc/selinux/config and set:

   SELINUX=permissive

   Reboot the server for the change to take effect.

    

Disabling SELinux reduces system security. Consider creating a custom SELinux policy to allow execution if you must keep SELinux enabled.

Cause

Exit status 126 in systemd indicates that the command was found but is not executable. This typically points to a permission issue on the script or binary that systemd is attempting to launch.

In this context of Talend Remote Engine, this is often caused by:

Cause A

Incorrect file permissions on the service wrapper scripts or binaries.

Cause B

SELinux policies prevents execution of the wrapper script or service file.

Environment

• Talend Cloud 
• Talend Studio 

The following release notes cover the Qlik PostgreSQL installer (QPI) version 1.2.0 to 2.0.0.

Content

• What's New
• 2.0.0 May 2025 Release Notes
• Known Limitations (2.0.0)
• 1.4.0 December 2023 Release Notes
• Known Limitations (1.4.0)
• 1.3.0 May 2023Release Notes
• Known Limitations (1.3.0)
• 1.2.0 Release Notes

What's New

• The PostgreSQL version used by QPI has been updated to 14.17
• An upgrade of an already upgraded external instance is now possible
• Support for an upgrade of the embedded 14.8 database was added
• Silent installs and upgrades are supported beginning with Qlik PostgreSQL Installer 1.4.0 and later. QPI can now be used with silent commands to install or upgrade the PostgreSQL database. (SHEND-973)

2.0.0 May 2025 Release Notes

Known Limitations (2.0.0)

• Rollback is not supported.
• The database size is not checked against free disk space before a backup is taken.
• Windows Server 2012 R2 does not support PostgreSQL 14.8. QPI cannot be used on Windows Server 2012 R2.
• QPI will only upgrade a PostgreSQL instance that only contains Qlik Sense Databases (QSR, SenseServices, QSMQ, Licenses, QLogs, etc.)

1.4.0 December 2023 Release Notes

Known Limitations (1.4.0)

• QB-24990: Cannot upgrade PostgreSQL if Qlik Sense was installed in a custom directory such as D:\Sense. See Qlik PostgreSQL Installer (QPI): No supported existing Qlik Sense PostgreSQL database found.
• Rollback is not supported.
• Database size is not checked against free disk space before a backup is taken.
• QPI can only upgrade bundled PostgreSQL database listening on the default port 4432. Using QPI to upgrade a standalone database or a database previously unbundled with QPI is not supported.
• Cannot migrate a 14.8 embedded DB to standalone
• Windows Server 2012R2 does not support PostgreSQL 14.8. QPI cannot be used on Windows Server 2012R2.

1.3.0 May 2023 Release Notes

A QlikView reload or Distribution task fails error message

System.IO.IOException: The process cannot access the file because it is being used by another process.

The task log will shows a successful reload but indicates the reloaded document can not be saved:

Information: Reload finished successfully
Information: Start saving document
Error: The sourcedocument failed to save.. Exception=System.IO.IOException: The process cannot access the file because it is being used by another process.
Error:    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
Error:    at System.IO.File.InternalReplace(String sourceFileName, String destinationFileName, String destinationBackupFileName, Boolean ignoreMetadataErrors)
Error:    at QVBWrapper.Document.ReplaceSourceFile(ILogBucket logBucket, String replaceFromFileName, String replaceToFilename)
Error:    at QVBWrapper.Document.Save(ILogBucket i_LogBucket, String i_SaveAsFileName)

File Access

After a successful reload of a QlikView Application (QVW) file, the old file is removed and replaced with a new file containing the new data. The error message indicates either that the old file could not be removed, or that file access is too slow to process the request. 

To identify if file access is too slow due to network latency, or due to disk read/write delays, review QlikView and its backend File Share System for details on the importance of fast file access, and how to monitor file shares.
In addition. 

In the case of file locks caused by processes locking the original file, the most common processes include:

• Anti-virus software
• Backup software
• QlikView services

The recommendation is to exclude QlikView folders from anti-virus scan. See What QlikView folder/files to exclude for anti-virus scanning for more details. 
Backup of QVW files must be planned and scheduled so that they do not interfere with QlikView tasks, to avoid file access conflicts.

The issue in general can only be resolved by identifying the process that is locking the file. See Process Monitoring - How to find locked files and the processes locking them for more details on how to identify the process that locks the QVW file.

Corrupted File

The file was previously corrupted.

1. Replace the QlikView document with a working copy
2. Rebuild the QlikView document from its project folder

You may be expericing some performance issues related to the task execution time or resuming file takes large storage problem. It is possible to reduce the resuming and recovery Logs in JobServer to solve them.

As the -Drecovery.enabled parameter is primarily used to control the creation and use of the recovery file, Setting it to false disables the feature and  meaning no resuming file will be generated or used. If you want to use recovery, you would leave this parameter enabled or set it to true (which is the default behavior)

This article briefly introuduces how to reduce job's resuming file if resuming file takes large storage

HOW TO

1. Stop Job Server.
2. Open the Job Server launch script start_rs.bat or start_rs.sh file and set the JVM arguments

   set MY_JVM_ARGS= -Drecovery.enabled=false    The default behavior is true

3. Restart Job Server.

Environment

• Talend Administration Center 
• Talend Data Integration 

In this article, we walk you through the requirements and process of how to upgrade and unbundle an existing Qlik Sense Repository Database (see supported scenarios) as well as how to install a brand new Repository based on PostgreSQL. We will use the Qlik PostgreSQL Installer (QPI).

For a manual method, see How to manually upgrade the bundled Qlik Sense PostgreSQL version to 12.5 version.

Using the Qlik Postgres Installer not only upgrades PostgreSQL; it also unbundles PostgreSQL from your Qlik Sense Enterprise on Windows install. This allows for direct control of your PostgreSQL instance and facilitates maintenance without a dependency on Qlik Sense. Further Database upgrades can then be performed independently and in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions. See How To Upgrade Standalone PostgreSQL.

Index

• Supported Scenarios
• Upgrades
• New installs
• Requirements
• Known limitations
• Installing anew Qlik Sense Repository Database using PostgreSQL
• Qlik PostgreSQL Installer - Download Link
• Upgrading an existing Qlik Sense Repository Database
• The Upgrade
• Next Steps and Compatibility with PostgreSQL installers
• How do I upgrade PostgreSQL from here on?
• Troubleshooting and FAQ
• Related Content

Video Walkthrough

Video chapters:

• 01:02 - Intro to PostgreSQL Repository
• 02:51 – Prerequisites
• 03:24 - What is the QPI tool?
• 05:09 - Using the QPI tool
• 09:27 - Removing the old Database Service
• 11:27 - Upgrading a stand-alone to the latest release
• 13:39 - How to roll-back to the previous version
• 14:46 - Troubleshooting upgrading a patched version
• 18:25 - Troubleshooting upgrade security error
• 21:15 - Additional config file settings

Supported Scenarios

Upgrades

The following versions have been tested and verified to work with QPI (1.4.0): 

Qlik Sense February 2022 to Qlik Sense November 2023.

If you are on a Qlik Sense version prior to these, upgrade to at least February 2022 before you begin.  

Qlik Sense November 2022 and later do not support 9.6, and a warning will be displayed during the upgrade. From Qlik Sense August 2023 a upgrade with a 9.6 database is blocked.

New installs

The Qlik PostgreSQL Installer supports installing a new standalone PostgreSQL database with the configurations required for connecting to a Qlik Sense server. This allows setting up a new environment or migrating an existing database to a separate host.

Requirements

• Review the QPI Release Notes before you continue

• Using the Qlik PostgreSQL Installer on a patched Qlik Sense version can lead to unexpected results. If you have a patch installed, either:

  • Uninstall all patches before using QPI (see Installing and Uninstalling Qlik Sense Patches) or
  • Upgrade to an IR release of Qlik Sense which supports QPI

• The PostgreSQL Installer can only upgrade bundled PostgreSQL database listening on the default port 4432.
• The user who runs the installer must be an administrator.
• The backup destination must have sufficient free disk space to dump the existing database  
• The backup destination must not be a network path or virtual storage folder. It is recommended the backup is stored on the main drive. 
• There will be downtime during this operation, please plan accordingly
• If upgrading to PostgreSQL 14 and later, the Windows OS must be at least Server 2016

Known limitations

• Cannot migrate a 14.8 embedded database to a standalone
• Using QPI to upgrade a standalone database or a database previously unbundled with QPI is not supported.
• The installer itself does not provide an automatic rollback feature.

Installing a new Qlik Sense Repository Database using PostgreSQL

1. Run the Qlik PostgreSQL Installer as an administrator 
   
   
2. Click on Install 
   
   
3. Accept the Qlik Customer Agreement 
   
   
4. Set your Local database settings and click Next. You will use these details to connect other nodes to the same cluster. 
   
   
5. Set your Database superuser password and click Next
   
   
6. Set the database installation folder, default: C:\Program Files\PostgreSQL\14

   Do not use the standard Qlik Sense folders, such as C:\Program Files\Qlik\Sense\Repository\PostgreSQL\ and C:\Programdata\Qlik\Sense\Repository\PostgreSQL\.

7. Set the database data folder, default: C:\Program Files\PostgreSQL\14\data

   Do not  use the standard Qlik Sense folders, such as C:\Program Files\Qlik\Sense\Repository\PostgreSQL\ and C:\Programdata\Qlik\Sense\Repository\PostgreSQL\.

8. Review your settings and click Install, then click Finish
   
   
9. Start installing Qlik Sense Enterprise Client Managed. Choose Join Cluster option.
   
   The Qlik PostgreSQL Installer has already seeded the databases for you and has created the users and permissions. No further configuration is needed.
   
   
10. The tool will display information on the actions being performed. Once installation is finished, you can close the installer.  
    
    If you are migrating your existing databases to a new host, please remember to reconfigure your nodes to connect to the correct host. How to configure Qlik Sense to use a dedicated PostgreSQL database 
    

Qlik PostgreSQL Installer - Download Link

Download the installer here.

Qlik PostgreSQL installer Release Notes

Upgrading an existing Qlik Sense Repository Database

The following versions have been tested and verified to work with QPI (1.4.0): 

February 2022 to November 2023.

If you are on any version prior to these, upgrade to at least February 2022 before you begin.  

Qlik Sense November 2022 and later do not support 9.6, and a warning will be displayed during the upgrade.  From Qlik Sense August 2023 a 9.6 update is blocked.

The Upgrade

1. Stop all services on rim nodes 
   
   
2. On your Central Node, stop all services except the Qlik Sense Repository Database 
   
   
   
   
3. Run the Qlik PostgreSQL Installer. An existing Database will be detected. 
   
   
   
   
4. Highlight the database and click Upgrade 
   
5. Read and confirm the (a) Installer Instructions as well as the Qlik Customer Agreement, then click (b) Next.
   
   
   
   
6. Provide your existing Database superuser password and click Next.
   
   
7. Define your Database backup path and click Next.
   
   
   
   
8. Define your Install Location (default is prefilled) and click Next.
   
   
   
   
9. Define your database data path (default is prefilled) and click Next.
   
   
    
10. Review all properties and click Upgrade.
    
    The review screen lists the settings which will be migrated. No manual changes are required post-upgrade.
    
    
    
    
11. The upgrade is completed. Click Close.
     
12. Open the Windows Services Console and locate the Qlik Sense Enterprise on Windows services.
    
    You will find that the Qlik Sense Repository Database service has been set to manual. Do not change the startup method. 
    
    
    
    You will also find a new postgresql-x64-14 service. Do not rename this service. 
    
    
    

     

13. Start all services except the Qlik Sense Repository Database service. 
    
    
14. Start all services on your rim nodes.
    
    
15. Validate that all services and nodes are operating as expected. The original database folder in C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\X.X_deprecated
    

     

16. Uninstall the old Qlik Sense Repository Database service.

    This step is required. Failing to remove the old service will lead the upgrade or patching issues.

    1. Open a Windows File Explorer and browse to C:\ProgramData\Package Cache
       
       
    2. From there, search for the appropriate msi file.
       
       
       
       If you were running 9.6 before the upgrade, search PostgreSQL.msi
       If you were running 12.5 before the upgrade, search PostgreSQL125.msi
       
       
    3. The msi will be revealed.
       
       
    4. Right-click the msi file and select uninstall from the menu.
       
       
       
       
    
    
17. Re-install the PostgreSQL binaries. This step is optional if Qlik Sense is immediately upgraded following the use of QPI. The Sense upgrade will install the correct binaries automatically. 
    

    Failing to reinstall the binaries will lead to errors when executing any number of service configuration scripts. 

    If you do not immediately upgrade:
    
    
    1. Open a Windows File Explorer and browse to C:\ProgramData\Package Cache
       
       
    2. From there, search for the .msi file appropriate for your currently installed Qlik Sense version
       
       
       
       For Qlik Sense August 2023 and later: PostgreSQL14.msi
       Qlik Sense February 2022 to May 2023: PostgreSQL125.msi
       
       
    3. Right-click the file
    4. Click Open file location
       
       
       
       
    5. Highlight the file path, right-click on the path, and click Copy
       
       
       
       
    6. Open a Windows Command prompt as administrator
       
       
    7. Navigate to the location of the folder you copied
       
       Example command line: 
       
       cd C:\ProgramData\Package Cache\{GUID}
       
       Where GUID is the value of the folder name.
       
       
    8. Run the following command depending on the version you have installed: 
       
       Qlik Sense August 2023 and later
       
       msiexec.exe /qb /i "PostgreSQL14.msi" SKIPINSTALLDBSERVICE="1" INSTALLDIR="C:\Program Files\Qlik\Sense"
       
       Qlik Sense February 2022 to May 2023
       
       msiexec.exe /qb /i "PostgreSQL125.msi" SKIPINSTALLDBSERVICE="1" INSTALLDIR="C:\Program Files\Qlik\Sense"
       
       This will re-install the binaries without installing a database. If you installed with a custom directory adjust the INSTALLDIR parameter accordingly. E.g. you installed in D:\Qlik\Sense then the parameter would be INSTALLDIR="D:\Qlik\Sense".
       
       
18. Finalize the process by updating the references to the PostgreSQL binaries paths in the SetupDatabase.ps1 and Configure-Service.ps1 files. For detailed steps, see Cannot change the qliksenserepository password for microservices of the service dispatcher: The system cannot find the file specified.

If the upgrade was unsuccessful and you are missing data in the Qlik Management Console or elsewhere, contact Qlik Support.  

Next Steps and Compatibility with PostgreSQL installers

Now that your PostgreSQL instance is no longer connected to the Qlik Sense Enterprise on Windows services, all future updates of PostgreSQL are performed independently of Qlik Sense. This allows you to act in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions.

Your PostgreSQL database is fully compatible with the official PostgreSQL installers from https://www.enterprisedb.com/downloads/postgres-postgresql-downloads. 

How do I upgrade PostgreSQL from here on?

See How To Upgrade Standalone PostgreSQL, which documents the upgrade procedure for either a minor version upgrade (example: 14.5 to 14.8) or a major version upgrade (example: 12 to 14). Further information on PostgreSQL upgrades or updates can be obtained from Postgre directly.

Troubleshooting and FAQ

1. If the installation crashes, the server reboots unexpectedly during this process, or there is a power outage, the new database may not be in a serviceable state. Installation/upgrade logs are available in the location of your temporary files, for example:
   
   C:\Users\Username\AppData\Local\Temp\2
   
   A backup of the original database contents is available in your chosen location, or by default in:
   
   C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\backup\X.X
   
   The original database data folder has been renamed to:
   
   C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\X.X_deprecated
   
   
2. Upgrading Qlik Sense after upgrading PostgreSQL with the QPI tool fails with:
   
   This version of Qlik Sense requires a 'SenseServices' database for multi cloud capabilities. Ensure that you have created a 'SenseService' database in your cluster before upgrading. For more information see Installing and configuring PostgreSQL.
   
   See Qlik Sense Upgrade fails with: This version of Qlik Sense requires a _ database for _.
   
   To resolve this, start the postgresql-x64-XX service.

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support. The video in this article was recorded in a earlier version of QPI, some screens might differ a little bit.

Related Content

Qlik PostgreSQL installer version 1.3.0 Release Notes  
Techspert Talks - Upgrading PostgreSQL Repository Troubleshooting
Backup and Restore Qlik Sense Enterprise documentation
Migrating Like a Boss
Optimizing Performance for Qlik Sense Enterprise
Qlik Sense Enterprise on Windows: How To Upgrade Standalone PostgreSQL
How-to reset forgotten PostgreSQL password in Qlik Sense
How to configure Qlik Sense to use a dedicated PostgreSQL database
Troubleshooting Qlik Sense Upgrades 

Qlik Sense allows for Data Encryption for Qlik Sense Apps (QVF) and Data (QVD) Files.

Qlik Sense Engine can encrypt data by using a data encryption key (DEK), which is generated from a certificate-based key encryption key (KEK). The DEK is unique to each encrypted file and is stored alongside the encrypted data (inside the QVF and QVD files) in an encrypted format using RSA. The industry standard AES-256 GCM is used as the data encryption algorithm. Decryption requires access to the same certificate as used during encryption, which mean that the KEK certificate thumbprint in the certificate store must match the thumbprint used for the DEK generation. 

Instructions

Data encryption can be enabled in the Service Cluster configuration of the Qlik Sense Management Console. See the Online Help at Qlik Sense Enterprise on Windows > Administer Qlik Sense Enterprise on Windows > Managing a Qlik Sense Enterprise on Windows site > QMC resources overview > Service cluster > Data encryption

Detailed Instructions with Example

This is a simple example of how to explore data encryption in Qlik Sense Enterprise on Windows.  

1. Create a new sample app, like the attached ascii-table.qvf
   1. Add a simple script to generate an ASCII table

      ASCII:
      Load
       if(RecNo()>=65 and RecNo()<=90,RecNo()-64) as Num,
       Chr(RecNo()) as AsciiAlpha,
       RecNo() as AsciiNum
      autogenerate 255
       Where (RecNo()>=32 and RecNo()<=126) or RecNo()>=160 ;

   2. Create a folder connection to a folder where the QVD can be written
   3. Generate QVD of a data table. Note, change the lib:// reference to match a valid data connection

      STORE ASCII INTO [lib://MyData (domain_administrator)/ascii.qvd] (QVD);

2. Reload the app
3. Copy the app and qvd files to allow comparison after enabling encryption 
   1. Default app location: C:\ProgramData\Qlik\Sense\Apps 
   2. The QVD location is per the app's folder data connection path
4. Create a certificate to test, following Encryption Certificates.
   1. Run Powershell as the Qlik Sense service account or log in to Windows as the Qlik Sense service account.
      
      

      Doing this with the correct account is crucial to make the generated certificate available to the service account.

      
      
      
   2. Generate a self-signed certificate to use as an encryption key
      
      Windows Server 2016
      
      

      New-SelfSignedCertificate -Subject "QlikSenseDataEncrytion" `
                                -KeyAlgorithm RSA `
                                -KeyLength 4096 `
                                -Provider "Microsoft Software Key Storage Provider" `
                                -KeyExportPolicy ExportableEncrypted `
                                -CertStoreLocation "cert:\CurrentUser\My"

      
      Windows Server 2012 R2
      
      

      New-SelfSignedCertificate -DnsName "QlikSenseDataEncrytion" `
                                -CertStoreLocation "cert:\CurrentUser\My"
      

   3. Validate that cert is available 
5. Get the cert thumbprint from generation result. This thumbprint can be used as key-encryption key (KEK) by Qlik Sense. 
   
   
   
   Note: When copying the certificate thumbprint, an invisible character may be added at the beginning of the certificate thumbprint. Verify the thumbprint before executing the command in PowerShell:
   
   
   
   
6. Enable encryption in Service Cluster settings per Data Encryption
   
   
   
   
7. Restart Qlik Sense Engine service
8. Reload the app to generate data encryption keys (DEK) for encrypted QVF and QVD files. 
   
   

Disable Encryption

Files remain encrypted after disabling encryption until the next following app reload or QVD generation. 

1. Disable encryption in QMC > Service Cluster
   1. Uncheck both encryption options
   2. Remove encryption key
      
2. Restart Qlik Sense Engine Service on all nodes
3. Complete a full successful app reload cycle, including QVD generators
   1. All QVF files have been decrypted
   2. All QVD files have been decrypted
      
      

File Comparison

Compare the unencrypted and encrypted files to validate successful encryption

Qlik Sense app (.qvf) file is a binary file, which makes it harder to visually confirm the encryption effect.

Encrypted app files have multiple references to ciphertext, which in turn refers to secrets used for the encryption.

App file without encryption has no such references. The cypher text portion of QVF represents the data encryption key (DEK) used to encrypt the app data and bookmarks. 

Encrypted data (.qvd) files has "Encryption Info" defined, which includes the data encryption (DEK) references.
Data part is also significantly different, even though both version have exactly the same static data content. 

Encryption Scope

• Encryption only applies at rest, meaning when stored on disk.  
• Data in memory is not encrypted
• Exported app (QVF) file is not encrypted
• Encrypted file (from apps storage folder) can not be imported in an other Qlik Sense instance.
  The import fails as the importing server is unable to parse the encrypted file. 
  
  

Related Content:

Qlik Sense on Windows: Data Encryption Key Rotation
Using Server Certificates for Data Encryption

This video will demonstrate how to install and configure Qlik-CLI for SaaS editions of Qlik Sense.

Content:

• Requirements
• Installation method One
• Installation Method Two
• Enable the Completion Feature (optional, but useful)
• Use Qlik-CLI
• Related Content
• Transcript

Requirements

• The user has access to a Qlik Cloud tenant
• The user has a professional license assigned in the tenant
• The user has the developer role assigned (required for API access)
• An API key can be obtained

Installation method One

1. Download Qlik-CLI from GitHub
2. Copy the executable to a local directory
3. Add the Qlik-CLI as an Environment Variable Path. 
   1. Open your Windows Control Panel
   2. Navigate to System 
   3. Open Advanced system settings in the leftmost menu
   4. Click Environment Variables 
   5. Locate Path in the User's System variables 
   6. Click Edit
   7. Click New
   8. Add the path to Qlik-CLI for example C:\Tools\Qlik-QLI\Modify the User Environment. Add path to Qlik-CLI. ie "C:\Tools\Qlik-QLI\"
   9. Confirm with OK until all windows are closed
4. Confirm the Qlik-CLI location path to verify it the installation completed successfully 
   1. Open PowerShell
   2. Execute:
      

      get-command qlik​

   3. Closer PowerShell

Installation Method Two

1. Download and install Chocolatey as documented on chocolatey.org.
2. Open PowerShell
3. Execute:
   

   choco install qlik-cli

4. This completes the installation.

Enable the Completion Feature (optional, but useful)

1. Verify if a PowerShell profile exists or create a new one
   1. Open PowerShell
   2. Execute:
      

      if ( -not (Test-Path $PROFILE) ) {
         echo "" > $PROFILE
      }​

2. Open the profile with the command notepad $PROFILE:
   

   qlik completion ps > "./qlik_completion.ps1" # Create a file containing the powershell completion.
   . ./qlik_completion.ps1                      # Source the completion.​

3. Save it. 
4. Restart PowerShell

Use Qlik-CLI

Advanced and additional instructions as seen in the video can be found at Qlik-CLI on Qlik.Dev. Begin with Get Started.

After adding a Global Rule to rename the target Schema (example: from PUBLIC to QLIK_TEST), the task may fail with an error indicating it was still trying to use the previous (example: PUBLIC) schema.

Reviewing the task JSON file to check the Global Rule definition shows no issues.

However, messages in the task logs indicate the table being created in the PUBLIC schema is not a task table. The table is an attrep change table:

00002308: 2024-03-25T16:51:38 [METADATA_MANAGE ]E: Failed (retcode -1) to execute statement: 'create or replace file format "PUBLIC"."ATTREP_FF_DEV_OS_STG_c1a25c9f_deef_f647_817b_db90dadf622b"

Resolution

Since the table mentioned in the error is not the actual target table, but a control table the resolution is to configure the control tables screen to also use the new schema (example: QLIK_TEST).

Screen Shot of the Control Tables Settings after setting schema to QLIK_TEST:

Related Content

Control Tables
Global Rules

When attempting to publish a standard DI job to Talend Management Console or an external artifact repository (regardless of if using Talend Studio or Talend CICD), you may be encountering the following error:

Caused by: org.talend.ipaas.services.inventory.InventoryException: workspace-service: failed to create artifact - Cannot publish artifact with the job type='data_service' because in one or more workspaces exists the same artifact with the different job type.

Other jobs may be able to publish to the same locations, even in the same project, without any specific issues.

Resolution

Two suggested solutions

• Put a tRESTClient (any ESB component) into the concurrent job, the “Build Type” combo box will be shown again, then select “Standalone” job, after that remove the tRESTClient, and save the job.
• Manually remove the BUILD_TYPE line from the job properties file.

  <additionalProperties xmi:id="_AVcxxO-mEe-Txxx_U4MkQ" key="BUILD_TYPE" value="OSGI"/>

Do not forget to re-publish your job at last

Cause

The issue, in many cases, may be an invalid entry within the job properties file, which still lists the job type as "Talend Runtime(OSGI)".
For this concurrent DI job, perhaps it was saved or exported at that time when there was a ESB component inside (Automatic Detective for build type). That’s why the job properties file had this line:

<additionalProperties xmi:id="_AVcxxO-mEe-Txxx_U4MkQ" key="BUILD_TYPE" value="OSGI"/>

During the publish-to-cloud via Talend studio or Talend CICD, the JobType will be set to “data_service” due to this wrongly set of BUILD_TYPE in the job properties file. With that invalid entry, the job cannot be used as a standalone job or microservice.

Environment

• Talend Studio 
• Talend Cloud 

Currently, in Qlik Application Automation it is not possible to export more than 100,000 cells using the Get Straight Table Data block.

Content:

• Full Automation
• Steps to set up the variables
• Automation structure

To overcome this limit, the workaround is to export records in batches from the Qlik Sense straight table to the cloud storage platform of your choice. The prerequisite is to have a unique numerical field in your dataset. If you don't have the unique field in your dataset, you can add it using RowNo() function in the load script as shown below. This counts the rows in the dataset.

In this example, we will export data from the Qlik Sense straight table to Dropbox as a CSV file.

You can also find an exported version of this automation and application attached to this article. More information on importing automation can be found here.

Full Automation

Automation Part 1

Automation Part 2

Automation Part 3

Steps to set up the variables

1. vAppId: Variable used to define your app ID.
2. vSheetId: Variable used to define your sheet ID.
3. vTableId: Variable used to define your straight table ID.
4. vFieldName: Variable used to define the field name on which the straight table data should be filtered based on the selection applied using the Select Field Value By Query block.

Automation structure

1. Add the Create Measure block to the automation and configure the Measure Expression parameter by providing an expression that calculates the number of rows in the dataset. In this example, I have added RowNo( ) as a unique field to the dataset, so the expression used is count([RowNo( )]).
2. Add the Get Measure block to get the number of rows in the dataset.
3. Store the number of rows obtained from the previous step in the vNRecords variable.
4. Delete the measure created using the Delete Measure block.
5. Add the vNLoops variable to calculate the number of loops that can be used to export straight table data in batches. Please note the formula used in the variable. Edit it to suit your needs. Change the 10000 value as you want but keep in mind that too many records could exceed the Get Straight Table Data block's 100,000 cells limit.
   
   
   
   
6. The Custom Code block creates a list with items 1 to n where n is the number of loops calculated in the previous step.
7. Add Create File on Dropbox block to create a new CSV file in Dropbox. You can select any other cloud storage connector, just remember to configure the relevant value in the Path parameter. The Qlik Sense straight table data will be exported to this file.
8. Add the Select Field Values By Query block to apply selection to the unique identifier created in the dataset to fetch only one record from the Get Straight Table Data block.
9. Add Write Line To File on Dropbox block to add straight table keys as column headers to the CSV file.
10. Add Loop block to loop over the list created using the Custom Code block.
11. Add Condition block to check if it's iterating over the 1st item, if yes set vRecordsStart variable value to 1, else set vRecordsStart variable value to vRecordsEnd variable. The vRecordsEnd variable represents the last record that needs to be fetched from the straight table during each iteration. Please note the formula used in the variable. The value under the Value 1 parameter in the Multiply formula should match the value provided in the Divide formula of the vNLoops variable.
    
    
    
    
12. Add the Select Field Values By Query block to apply selection to the unique identifier created in the dataset to fetch the records between vRecordsStart and vRecordsEnd variables from the Get Straight Table Data block.
13. Add Write Line To File on Dropbox block within the Get Straight Table Data block which writes each straight table data into the CSV file. Configure it to use the output from the Get Straight Table Data block as the Data parameter.
14. Add Save And Close File on Dropbox block and configure the file parameter to use the CSV file we have created.

Environment

• Qlik Cloud
• Qlik Automate