Search code examples
c++compressionzlib

Can zlib remove padding without compressing the rest of the file?


I'm looking for a way to apply a specific compression strategy using zlib-1.2.5. I need to force zlib to pack a file without actually compressing it. The only part I want to compress is 0 padding at the end of the file, so that the output file does not have it. The size of input files ranges from 1MB to 1GB, padding is 512B. Is this achievable with zlib?

Edit:

The code I'm working on is based on Unreal Engine 4: https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Runtime/Core/Private/Misc/Compression.cpp

static bool appCompressMemoryZLIB(void* CompressedBuffer, int32& CompressedSize, const void* UncompressedBuffer, int32 UncompressedSize, int32 BitWindow, int32 CompLevel)
{
    DECLARE_SCOPE_CYCLE_COUNTER(TEXT("Compress Memory ZLIB"), STAT_appCompressMemoryZLIB, STATGROUP_Compression);

    ensureMsgf(CompLevel >= Z_DEFAULT_COMPRESSION, TEXT("CompLevel must be >= Z_DEFAULT_COMPRESSION"));
    ensureMsgf(CompLevel <= Z_BEST_COMPRESSION, TEXT("CompLevel must be <= Z_BEST_COMPRESSION"));

    CompLevel = FMath::Clamp(CompLevel, Z_DEFAULT_COMPRESSION, Z_BEST_COMPRESSION);

    // Zlib wants to use unsigned long.
    unsigned long ZCompressedSize = CompressedSize;
    unsigned long ZUncompressedSize = UncompressedSize;
    bool bOperationSucceeded = false;

    // Compress data
    // If using the default Zlib bit window, use the zlib routines, otherwise go manual with deflate2
    if (BitWindow == 0 || BitWindow == DEFAULT_ZLIB_BIT_WINDOW)
    {
        bOperationSucceeded = compress2((uint8*)CompressedBuffer, &ZCompressedSize, (const uint8*)UncompressedBuffer, ZUncompressedSize, CompLevel) == Z_OK ? true : false;
    }
    else
    {
        z_stream stream;
        stream.next_in = (Bytef*)UncompressedBuffer;
        stream.avail_in = (uInt)ZUncompressedSize;
        stream.next_out = (Bytef*)CompressedBuffer;
        stream.avail_out = (uInt)ZCompressedSize;
        stream.zalloc = &zalloc;
        stream.zfree = &zfree;
        stream.opaque = Z_NULL;

        if (ensure(Z_OK == deflateInit2(&stream, CompLevel, Z_DEFLATED, BitWindow, MAX_MEM_LEVEL, Z_DEFAULT_STRATEGY)))
        {
            if (ensure(Z_STREAM_END == deflate(&stream, Z_FINISH)))
            {
                ZCompressedSize = stream.total_out;
                if (ensure(Z_OK == deflateEnd(&stream)))
                {
                    bOperationSucceeded = true;
                }
            }
            else
            {
                deflateEnd(&stream);
            }
        }
    }

    // Propagate compressed size from intermediate variable back into out variable.
    CompressedSize = ZCompressedSize;
    return bOperationSucceeded;
}

With params: input buffer size = 65kB, CompLevel=Z_DEFAULT_COMPRESSION, MAX_MEM_LEVEL=9, BitWindow=15


Solution

  • No, there is not.

    If all you want to do is strip the zeros at the end, then that is trivial. It's a few lines of code. You don't need zlib's 18,000 lines of code to do that.

    If you also want to restore those zeros to the file at the other end ("decompressing it"), that is trivial as well. Just send a count of the zeros that were removed, with that count in a few bytes appended to the end. On the other end, replace that count with that many zero bytes. Also a few lines of code.