Search code examples
compressiongzip

How to create a gzip file without actually compressing the file


I have several tools that expects a gzip file but a file that is created not gzipped, i'd like these tools to access the regular file without actually compressing it (this takes too much time and will happen later in the pipeline).

Is it possible to make it behave like a gzipped file so it can be handled by tools that expect gzip files?

The file is created with null bytes at the beginning so i can write any leading bytes directly to the file to make an empty compression gzip header, but i don't know what the header should be (if this is at all possible..)


Solution

  • Yes, you can create a gzip file without spending any time attempting to compress it. However you cannot do that simply with a prepended header. You would need to use stored blocks in the deflate stream, and you would need to compute the CRC-32 for the trailer.

    The deflate stream would be a series of blocks, where each block except the last is (in hex), 00 ff ff 00 00 followed by 65,535 bytes of the input. The last block would be 01 xx xx yy yy, where xx xx is the number of bytes remaining in little-endian order, and yy yy is the one's complement of xx xx. That is followed by the remaining bytes.

    The header can be a vanilla gzip header, 1f 8b 08 00 00 00 00 00 00 00. The trailer is cc cc cc cc nn nn nn nn, where the first four bytes is the CRC-32 of the uncompressed data, and the second are the number of bytes of uncompressed data (modulo 2^32 if 4GB or more), both in little-endian order.