Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW

Qlik Sense TLS Troubleshooting

No ratings
cancel
Showing results for 
Search instead for 
Did you mean: 
Zareh_T
Support
Support

Qlik Sense TLS Troubleshooting

Last Update:

Jun 16, 2021 8:43:21 AM

Updated By:

Sonja_Bauernfeind

Created date:

Jul 17, 2017 3:22:28 AM

Summary and Purpose

This document is aimed at providing steps to gather information to help identify TLS, Network and Certificate related issues. Follow the steps in order; although skip any steps to set up extra logging deemed not to be required. Cleaning out the Qlik certificates should be done to eliminate any problems with them causing TLS or other connection issues.

Steps are:

  1. Prepare the server/s
  2. Stop Qlik services
  3. Set up tracing, logging and clean out certificates
  4. Start Qlik services
  5. Test
  6. Stop Qlik services
  7. Stop logging and undo changes

 

1. Prepare the server(s): Set up a logging capture directory

  1. Stop all unnecessary Windows applications
  2. Create a directory C:/Temp
  3. Set permissions so that the Qlik Sense Services owner and the logged in user can write to it
    1. Right click > Properties > Security > Edit > select User > select ‘Full Control’ > Apply > OK
    2. To identify the Sense Services Owner
      1. Open Services > Right click on ‘Qlik Sense Repository Service > Properties > Log on
  4. Open a Command Prompt window with Administrative privileges
  5. Change to the C:/Temp directory, enter cd C:/Temp
  6. Run all commands from this Command Prompt Window

2. Stop Qlik Sense Services​

  1. Use the Windows Services gui to stop all Qlik Sense Services
    1. Or from the Command Prompt run net stop "Qlik Sense Repository Database"
    2. Select to stop all associated Services
    3. From the Command Prompt un net stop QlikSenseServiceDispatcher

3. Set up tracing, logging and clean out certificates

3a. Proxy Logging

Enable trace logging for the Proxy and Repository services.

  1. Save a copy of the Proxy.exe.config file. For example as Proxy.exe.config.orig
    1.  File is located in C:\Program Files\Qlik\Sense\Proxy
  2. In the Proxy.exe.config file add the following at the END of the file but before the </configuration> element:
<system.diagnostics> 
<trace autoflush="true" /> 
<sources> 
<source name="System.Net"> 
<listeners> 
<add name="System.Net"/> 
</listeners> 
</source> 
<source name="System.Net.HttpListener"> 
<listeners> 
<add name="System.Net"/> 
</listeners> 
</source> 
<source name="System.Net.Sockets"> 
<listeners> 
<add name="System.Net"/> 
</listeners> 
</source> 
<source name="System.Net.Cache"> 
<listeners> 
<add name="System.Net"/> 
</listeners> 
</source> 
</sources> 
<sharedListeners> 
<add 
name="System.Net" 
type="System.Diagnostics.TextWriterTraceListener" 
initializeData="c:\temp\proxy_trace.log" 
traceOutputOptions = "ProcessId, DateTime" 
/> 
</sharedListeners> 
<switches> 
<add name="System.Net" value="Verbose" /> 
<add name="System.Net.Sockets" value="Verbose" /> 
<add name="System.Net.Cache" value="Verbose" /> 
<add name="System.Net.HttpListener" value="Verbose" /> 
</switches> 
</system.diagnostics>

 

3b. Repository Logging

  1. Save a copy of the Repository.exe.config file. For example as Repository.exe.config.orig
    1. File is located in C:\Program Files\Qlik\Sense\Repository
  2. In the Repository.exe.config file add the following at the END of the file but before the </configuration> element:
<system.diagnostics> 
<trace autoflush="true" /> 
<sources> 
<source name="System.Net"> 
<listeners> 
<add name="System.Net"/> 
</listeners> 
</source> 
<source name="System.Net.HttpListener"> 
<listeners> 
<add name="System.Net"/> 
</listeners> 
</source> 
<source name="System.Net.Sockets"> 
<listeners> 
<add name="System.Net"/> 
</listeners> 
</source> 
<source name="System.Net.Cache"> 
<listeners> 
<add name="System.Net"/> 
</listeners> 
</source> 
</sources> 
<sharedListeners> 
<add 
name="System.Net" 
type="System.Diagnostics.TextWriterTraceListener" 
initializeData="c:\temp\repository_trace.log" 
traceOutputOptions = "ProcessId, DateTime" 
/> 
</sharedListeners> 
<switches> 
<add name="System.Net" value="Verbose" /> 
<add name="System.Net.Sockets" value="Verbose" /> 
<add name="System.Net.Cache" value="Verbose" /> 
<add name="System.Net.HttpListener" value="Verbose" /> 
</switches> 
</system.diagnostics>

Verbose trace info should now be written to c:\temp\proxy_trace.log and c:\temp\repository_trace.log respectively when Qlik Sense Services are restarted.

3c. Clean out Qlik Sense certificate

Note: All Qlik Sense services should be stopped before proceeding

- Clean out client certificate for Qlik Sense service user:

  1. Go to C:\Windows\System32
  2. Shift + Right click on mmc.exe and choose "Run as different user"
  3. Enter the credentials of the Windows account (service logon) running the Qlik Sense services
  4. Choose File->Add/Remove Snap-in...>Add->OK
  5. Delete "QlikClient" under Personal/Certificates

- Clean out root and service certificates:

  1. Go to C:\Windows\System32
  2. Right click on mmc.exe and choose "Run as Administrator"
  3. Choose File->Add/Remove Snap-in...>Add->select Computer account->Finish->OK
  4. Delete "[hostname]" issued by "[hostname]-CA" under "Personal/Certificates"
  5. Delete "[hostname]-CA" under "Trusted Root Certification Authorities/Certificates"

Note. If the user running the Qlik services is not a member of the local Administrators group, you'll need to perform the bootstrap procedure described here in the Qlik Sense Online Help section on Using an account without administrator privileges

 

- Adjust Trusted Issuers List

Certificate list is truncated during the TLS handshake.

If there is a large number of root certificates it might cause servers to hit the following issue: http://netsekure.org/2011/04/tls-client-authentication-and-trusted-issuers-list/ This may result in the Sense root certificate not being included in the trusted issuers list and thus we cannot find the matching client certificate to use.
This can be worked around by setting the following registry key:

  1. Run Registry Editor (regedt.exe) from the Command Prompt
  2. Change or add the key

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL
Value name: SendTrustedIssuerList Value type: REG_DWORD Value data: 0 (False)

 

TLS Settings and Logging

- Make sure TLS is not disabled

We should make sure that TLS is allowed on the server. Please make sure that none of the "Enabled" keys are present and set "0", or "DisabledByDefault" is present and set to "1" for the TLS protocol/s as described here: https://technet.microsoft.com/en-us/library/dn786418(v=ws.11).aspx#BKMK_SchannelTR_TLS10

- Enable Schannel (TLS handshake) logging

Note. All Qlik services should be stopped before proceeding.

  1. Using Registry Editor change value for "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\EventLogging" to "7"
  2. Observe the Schannel logs in Windows Event Viewer -> Windows Logs -> System. Filter on source "Schannel"
  3. Export the Windows System and Application logs

- Network Trace – do this last!

Run the below from the Command Prompt

netsh trace start capture=yes scenario=netconnection maxsize=2000 filemode=circular overwrite=yes report=no persistent=yes tracefile=c:/Temp/%computername%.etl

This creates 2 files (etl and .cab) with the computer name in the C:/temp directory
 
Note: Microsoft Message Analyzer is required to view the logs created. See the Appendix for suggested filter parameters

4. Start Qlik Sense Services

  1. Use the Windows Services gui to start all Qlik Sense Services
  2. Or from the Command Prompt run
    1. net start "QlikSenseProxyService"
    2. net start "QlikSenseEngineService"
    3. net start "QlikSensePrintingService"
    4. net start "QlikSenseSchedulerService"
    5. net start “QlikSenseServiceDispatcher”
  3. Allow a few minutes for all services to complete start up processes

After Starting Qlik Sense Services

- Network:

  1. From the Command Prompt run
    netsh http show iplisten > c:/Temp/iplisten.txt

Shows listening interfaces for all services, i.e. which IP addresses the Sense services are listening on (default is empty = all interfaces)

 

- Certificates:

Certificate information

  1. From the Command Prompt run
    netsh http show sslcert > c:/Temp/sslcert.txt

Captures which SSL certificates are bound to specific listening ports.

  1. From the Command Prompt run
    certutil -store My > c:/Temp/cert_store.txt

Captures certificate thumbprints which can be matched to the output of above command.

5. Complete testing

Complete whatever steps are needed to reproduce the problem.
Keep a record of the time, as closely as possible, for each step of the testing; or if it’s a short test the start and stop times.

Post Testing Steps

Gather logs and undo all changes made to the system.

6. Stop the Qlik Sense Services

  1. Use the Windows Services gui to stop all Qlik Sense Services
  2. Or from the Command Prompt run net stop "Qlik Sense Repository Database"
    1. Select to stop all associated Services
    2. From the Command Prompt un net stop “QlikSenseServiceDispatcher”

- Reset Sense Config files

  1. Change the Proxy.exe.config and Proxy.exe.config files to end in .trace
  2. Change the Proxy.exe.config.orig ans Proxy.exe.config.orig by removing the .orig

7. Stop Logging and Undo Changes - Stop the Network trace

  1. From the Command Prompt run netsh trace stop
  2. It may take a few minutes for this to complete creating the trace files

- Collect Logs and files​

  1. Export the Windows Event Logs to a file
    1. Open the Windows Event Viewer
    2. Expand the Windows Logs list > Right Click on System > Click on Save all Event As…
    3. Save to the C:/Temp directory as SystemLogs.evtx
    4. Repeat for the Application Log, save as ApplicationLog.evtx
  2. Compress the C:\Temp Directory
    1. Right click on the Temp folder
    2. Click on Send To > Select Compressed (zipped) folder

Note: The Repository and Network trace files can be quite large, so the zip file may need to be split. See your compression tool instructions on how to achieve that.

- Reset the registry keys

- Disable Schannel (TLS handshake) logging

Note. All Qlik services should be stopped before proceeding.

  1. Using Registry Editor change value for "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\EventLogging" to "0" or the previously set value

- TLS Settings

Reset the TLS registry key values if required

Appendix

 

- Microsoft Message Analyzer

Suggested Layout

Message Number, Timestamp, PayloadLength, Summary, ProcessName, SourePort, Destination Port
Adjust based the nature of the problem.

Example Filters

The * is for all field types. Filters can be by protocol or communications type, e.g. TCP, HTTP or LDAP. Use filters aimed at the type of problem.

  • *ProcessName == "Proxy.exe"
  • *ProcessName == "Repository.exe"
  • TCP.ProcessName == "Proxy.exe"
  • Microsoft_Windows_TCPIP.DestinationPort == 4242
  • Microsoft_Windows_TCPIP.SourcePort == 4242

 
Download from https://www.microsoft.com/en-au/download/details.aspx?id=44226

Combine filters by using ‘and’, ‘or’, etc.

Microsoft_Windows_WinHttp.ProcessName == "Repository.exe" or (Windows_Kernel_Trace.ProcessName == "Proxy.exe")
(*ProcessName == "Proxy.exe") And (*DestinationPort == 4242)

Other Tools that may be useful

3rd Party Security and Antivirus

McAfee interferes with the client authentication process.

There seems to be an issue with the McAfee security software that could interfere with client authentication process. It explicitly states that it's not affecting self-signed certs, but the symptoms can match customer situations and may be worth investigating with McAfee. https://kc.mcafee.com/corporate/index?page=content&id=KB87705
 

Labels (2)
Contributors
Version history
Last update:
‎2021-06-16 08:43 AM
Updated by: