Search code examples
c++compressionzlib

How to assume what will be the size after zlib uncompression?

I am making a simple C++ app that has to send compressed data to my API. The API fires a response at the app that is also compressed. I have to uncompress it. I am using zlib's uncompress function but I do not know how big the data is. Can some one help me with this problem? How can I calculate and set the size of the destination buffer?

Solution

  • I think the documentation is really clear about this

    ZEXTERN int ZEXPORT uncompress OF((Bytef *dest, uLongf *destLen,
                               const Bytef *source, uLong sourceLen));

    Decompresses the source buffer into the destination buffer. sourceLen is the byte length of the source buffer. Upon entry, destLen is the total size of the destination buffer, which must be large enough to hold the entire uncompressed data. (The size of the uncompressed data must have been saved previously by the compressor and transmitted to the decompressor by some mechanism outside the scope of this compression library.) Upon exit, destLen is the actual size of the uncompressed data.

    uncompress returns Z_OK if success, Z_MEM_ERROR if there was not enough memory, Z_BUF_ERROR if there was not enough room in the output buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete. In the case where there is not enough room, uncompress() will fill the output buffer with the uncompressed data up to that point.

    So zlib recommends sending over the uncompressed size along with the compressed stream.

    But we can also note the sentence

    In the case where there is not enough room, uncompress() will fill the output buffer with the uncompressed data up to that point.

    So you can include the length in the compressed message at the beginning. Then at your destination start uncompressing with a small buffer. It might not uncompress everything into the small buffer. But it will decompress enough for you to read the data length if you wrote it at the beginning. Then you can use that to allocate/resize your destination buffer and use uncompress again.

    Depending on your use-case this might be a good idea or not. If your message sizes don't vary much and the program is longer running it might just be better to just maintain one destination buffer and just grow that one as needed.