Find Answers
Qlik Community
Collaborate with over 60,000 Qlik technologists and members around the world to get answers to your questions, and maximize success.
Join Us
Knowledge
A Qlik View 11.20.x app can increase significally it's size in 12.x versions
Article Number: 000034764 | Last Modified: 2018/06/12Description
In the old row based data model (11.20), each join did a full sort of the resulting table. This was strictly not necessary, but in the row based model, it incurred little extra cost.
In the new column based data model (12.x), the join saves a lot of effort by only sorting the table(s) on the join columns. In the columnar model, it’s cheap to work on a subset of columns simply. The speeds up the joins considerably for most cases.
The result is logically the same, but the resulting tables are considerably less ordered. This can affect the compression applied to the data when written to file. In the worst cases, there’s many columns in the table with few and well-ordered values when sorted fully. A typical example would be a column with e.g. [OrderYear] or e.g. a classifying column like [UserType]=USER/ADMIN. When fully sorted, tables with many such columns can become very very predictable, e.g. the [OrderYear] column may have 10M rows with 2014, followed by 10M rows with 2015 etc. When only partially sorted, a column such as [OrderYear], which is unlikely to be a join column, can appear as a complete mix of numbers due to the joins that re-order it based on the join field(s).
As the QVW is still stored as row-based data, i.e. it’s writes the table as it would look in Excel, from left to right, top to bottom, in chunks of data, a few unordered columns can significantly reduce the uniformity of data within each chuck, resulting in much lower compression ratio. For really bad cases this can be a factor of 10 almost.
In the new column based data model (12.x), the join saves a lot of effort by only sorting the table(s) on the join columns. In the columnar model, it’s cheap to work on a subset of columns simply. The speeds up the joins considerably for most cases.
The result is logically the same, but the resulting tables are considerably less ordered. This can affect the compression applied to the data when written to file. In the worst cases, there’s many columns in the table with few and well-ordered values when sorted fully. A typical example would be a column with e.g. [OrderYear] or e.g. a classifying column like [UserType]=USER/ADMIN. When fully sorted, tables with many such columns can become very very predictable, e.g. the [OrderYear] column may have 10M rows with 2014, followed by 10M rows with 2015 etc. When only partially sorted, a column such as [OrderYear], which is unlikely to be a join column, can appear as a complete mix of numbers due to the joins that re-order it based on the join field(s).
As the QVW is still stored as row-based data, i.e. it’s writes the table as it would look in Excel, from left to right, top to bottom, in chunks of data, a few unordered columns can significantly reduce the uniformity of data within each chuck, resulting in much lower compression ratio. For really bad cases this can be a factor of 10 almost.
Resolution
Working as design
Get Answers
Find Answers
Get Support
Have a Question?
Search Qlik's Support Knowledge database or request assisted support for highly complex issues.
Submit a case
Talk with a Representative
Critical Issues
Experiencing a serious issue, please contact us by phone. For Data Integration related issues please refer to your onboarding documentation for current phone number.
Call Us