Byte Order Mark(BOM)

[Andre_Sostizzo]

Open a TXT file generated by QlikView or Qlik Sense (it can be logs or files created by "STORE" command) in a HEX editor(such as HxD), the first 3 bytes are 0xEF 0xBB 0xBF, although nothing is shown when opening it in a normal text editor.

Sample (QlikView):
 

 

Environment:

QlikView and Qlik Sense any version 

BOM is introduced with the intention of identifying the encoding of a "stream" being used, so that the receiver can decode the data correctly.

However, BOM may cause problem if "sender" and "receiver" are not having any "agreement" of its usage. For example if the receiver is not "expecting" BOM, then it may process BOM as actual data, which BOM is not.

In most cases, all Unicode recognized software should by standard interpret BOM as "Zero Width No-Break Space". That is why when viewing in Notepad, BOM shows nothing.

References:

http://tools.ietf.org/html/rfc3629#section-6
http://www.unicode.org/versions/Unicode6.0.0/ch16.pdf (page 18)

Resolution:

These 3 particular bytes are called "Byte order mark(BOM)". It basically means this document is encoded in Unicode. When using UTF-8, BOM is 0xEF 0xBB 0xBF.

This is an expected behavior because by default QlikView stores TXT files in UTF-8 encoding. In fact BOM is widely implemented by other product such as Notepad in Windows and Notepad++.