Search code examples
sql-serverssasbusiness-intelligenceqlikview

The QVW File Size Is Larger While Using Cube Data


I've made two QlikView (two .qvw files) reports which have almost the same content. One loads data from Data Warehouse using 'Microsoft OLE DB Provider for SQL Server'. Another one load data from Cube using 'Microsoft OLE DB Provider for Analysis Services'.

However, the file's size with Cube data is much larger than the file with DW data. I want to know why this happened.

Thanks for attention.


Solution

  • Define almost the same content :)

    QlikView/QlikSense file size are depending on the number of the columns and the distinct values in them.

    In general Qlik do not store all the data as it is. Instead it stores the data as lists of distinct values and memory pointers and thats how the data model is build.

    For example:

    If you have two fields in the database (both with 1M rows) - one with timestamp values and the other with only date values.

    In Qlik, the app with the timestamp values will be bigger than the date app because timestamp field have a lot of non distinct values (lets say 80% are non distinct) and date have a lot of distinct values (let say 80% are distinct) and Qlik stores only the distinct values in the background.

    In the example above you will end with timestamp file being saved with 800k rows (80% non distinct) and date file with 200k (80% distinct) rows although in the database having both fields with 1M rows.

    Im simplifying it, but in general - less fields with high number of distinct values will produce smaller files. Adding 1-2 more columns with a lot non-distinct values can make the file size quite different (especially with bigger datasets).

    Real example:

    • SO_47650951_1.qvw - contains 1 field with "1M" rows. The values in it are floor( rand() * 100) (1,2,3,4 ... 100) and file size is 968Kb
    • SO_47650951_2.qvw - contains 1 field with "1M" rows. The values in it are rand() (0.70505229476839, 0.22396470652893, 0.59824940375984 etc) and file size is 7,2Mb "