55 What is a QVD file

Опубликовано: 28 Сентябрь 2024
на канале: Abhishek Agarrwal

6,677

In this qliksense tutorial video I have talked about what is a qvd file and why it is used in qliksense. Also talked about some technical details like why it is able to compress data as well what is the structure of the qvd file in the backend.

#QlikSenseTutorial

A QVD (QlikView Data) file is a file containing a table of data exported from Qlik Sense. QVD is a native Qlik format and can only be written to and read by Qlik Sense or QlikView. The file format is optimized for speed when reading data from a script but it is still very compact. Reading data from a QVD file is typically 10-100 times faster than reading from other data sources.

QVD files can be read in two modes: standard (fast) and optimized (faster). The selected mode is determined automatically by the script engine.

There are some limitations regarding optimized loads. It is possible to rename fields, but any of the operations mentioned here will disable the optimized load and result in a standard load.

Any transformations on the fields that are loaded.
Using a where clause causing Qlik Sense to unpack the records.
Using Map on a field that is loaded.

Purpose of QVD files
QVD files can be used for many purposes and more than one may apply in any given situation. At least four major uses can be easily identified:

Increasing load speed
By buffering non-changing or slowly-changing blocks of input data in QVD files, script execution becomes considerably faster for large data sets.

Decreasing load on database servers
The amount of data fetched from external data sources can also be greatly reduced. This reduces the workload on external databases and network traffic. Furthermore, when several scripts share the same data, it is only necessary to load it once from the source database into a QVD file. Other apps can make use of the same data through this QVD file.

Consolidating data from multiple apps
With the binary script statement, data can be loaded from a single app into another app, but with QVD files a script can combine data from any number of apps. This makes it possible for apps to consolidate similar data from different business units, for example.

Incremental
In many common cases, the QVD functionality can be used for incremental load by loading only new records from a growing database.

Creating QVD files
A QVD file can be created in two ways:

Explicit creation and naming using the store command in the script. State in the script that a previously-read table, or part thereof, is to be exported to an explicitly-named file at a location of your choice.

For more information, see Store.

Automatic creation and maintenance from script. When you precede a LOAD or SELECT statement with the buffer prefix, Qlik Sense will automatically create a QVD file, which, under certain conditions, can be used instead of the original data source when reloading data.

For more information, see Buffer.

There is no difference between the resulting QVD files with regard to reading speed.

QVD format
A QVD file holds exactly one data table and consists of three parts:

Header.

Note: If the QVD file was generated with QlikView the header is a well-formed XML header (in UTF-8 char set) describing the fields in the table, the layout of the subsequent information and other metadata.
Symbol tables in a byte-stuffed format.
Actual table data in a bit-stuffed format.