Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW

Search Folder Growing

100% helpful (1/1)
cancel
Showing results for 
Search instead for 
Did you mean: 
Chris_Rice
Support
Support

Search Folder Growing

Last Update:

Jun 15, 2022 9:41:19 AM

Updated By:

Sonja_Bauernfeind

Created date:

Jul 9, 2018 8:24:10 AM

Qlik Sense Search Folder is growing in size.

Smart search is the global search tool in Qlik Sense, enabling you to search the entire data set in your app from any sheet in the app. See Using Smart Search on the official Qlik Online Help for details. 

This search is indexed to disk at %Share%/Apps/Search.

As time goes on, the index grows and commits more disk space. 
 

Environment: 

 

Resolution:


The contents of the Qlik Sense Search folders can be deleted to free up disk space. The content pertains to App ID's, so it would be best to remove those which are not often used. The only effect this would have is increasing the amount of time needed to complete a Smart Search for this same App in the future.

Qlik Sense can also be configured to purge the Search Folder on a schedule. The setting will need to be applied in the settings.ini of either the Qlik Sense Enterprise Server or the Qlik Sense Desktop. For information on how to edit the settings.ini file, reference How to modify Qlik Sense Engine's Settings.ini.
 

Qlik Sense Server %PROGRAMDATA%/Qlik/Sense/Engine/Settings.ini
Qlik Sense Desktop %USERPROFILE%/Documents/Qlik/Sense/Settings.ini

 

Name

ServerPurgeSearchIndexFileInterval

Default Value

3600

Description

  • -1 the search index folder is never purged.
  • 0 or negative (except -1), the purge happens constantly.
  • strictly positive value, the search index folder is purged each ServerPurgeSearchIndexFileInterval seconds.

The purge system removes the unused search index files. In addition, it removes the oldest used search index files until the search index folder is smaller than ServerSearchIndexFileDirectoryMaximumSizeInMBytes.

Name

ServerSearchIndexFileDirectoryMaximumSizeInMBytes

Default Value

-1

Description

Maximum size of the search index directory in megabytes. During the periodic purge, old indexes are removed until the total size of index files is below this threshold. If negative, do not limit the size of the directory.

 

Make sure to add both parameters mentioned above into the settings.ini, the cleaning will happen at the specified intervall but only after the size limit is reached.

Labels (1)
Comments
DWruck
Contributor
Contributor

Is there a log file created when the directory is pruned, and if so where is it created? Also is there a way to set the timing on the tasks?

Sonja_Bauernfeind
Digital Support
Digital Support

Hello @DWruck 

I will look into this for you (though I am going to be out of the office until next week, so expect an answer sometime then).

All the best,
Sonja 

Sonja_Bauernfeind
Digital Support
Digital Support

@DWruck 

I have set the Engine to log at information/debug level and was unable to find records of the purge. 

DWruck
Contributor
Contributor

@Sonja_Bauernfeind 

Thank you for that, is there a criteria list for how the files are removed?

Sonja_Bauernfeind
Digital Support
Digital Support

Hello @DWruck 

Can you be more specific on what you mean with what criteria you are missing?

The criteria that we have listed are:

ServerPurgeSearchIndexFileInterval removed them at the given interval, beginning with the oldest unused. ServerSearchIndexFileDirectoryMaximumSizeInMBytes removes them when the specified size is given, again based on the oldest not used.

All the best,
Sonja 

DWruck
Contributor
Contributor

@Sonja_Bauernfeind I am trying to figure out the order in which the files are removed from the search directory, is it date-based or just random deletion?

Sonja_Bauernfeind
Digital Support
Digital Support

@DWruck It goes by oldest (not used the longest) until the criteria for the cleanup is met (folder size smaller, etc).

davidhechler
Partner - Contributor II
Partner - Contributor II

Hi @Sonja_Bauernfeind,

First off, thanks for the detailed explanation of the solution.
Now I have the following question:
Is there a rule of thumb or correlation between app size and recommended size of the parameter ServerSearchIndexFileDirectoryMaximumSizeInMBytes? E. g. if the app has a size of 1.3 GB on disk, the ServerSearchIndexFileDirectoryMaximumSizeInMBytes should be set to at least 1300?

I had a customer who's biggest app had a size of around 1.3 GB, but there was an index file with more than 23 GB causing the hard disk to run out of space.

Best regards,
David

Sonja_Bauernfeind
Digital Support
Digital Support

Hello @davidhechler 

Let me look into this for you!

All the best,
Sonja 

Sonja_Bauernfeind
Digital Support
Digital Support

Hello @davidhechler 

The size of the files is not specifically tied to the "size" of the app, but is more tied to the number of fields. You can govern which fields to include in the index: How can I exclude fields from the Global Search tool in Qlik Sense Enterprise on Windows? 

As for the recommended size: there are no best practices around this, as the size can vary greatly. The only effect of limiting the size would be that the files are deleted more frequently, meaning the time needed to complete a Smart Search from this app may take longer the next time. 

I hope this helps!

All the best,
Sonja 

Version history
Last update:
‎2022-06-15 09:41 AM
Updated by: