Search code examples
zipcompressiondeflatedeflatestream

Deflating data from MSZIP format


I'm trying to read a compressed binary .x mesh file but my decompression is failing. The file is basically some directx header info and then a bunch of data in MSZIP format (i.e. 2 bytes are an int blockSize, 2 bytes are a "magic number" and then there are blockSize deflated bytes and then repeat until there's no more data) so for each block I'm just getting the compressed bytes and deflating like so-

internal static byte[] DecompressBlock(byte[] data) {
    var result = new List<byte>();
    var ms = new MemoryStream(data);
    var ds = new DeflateStream(ms, CompressionMode.Decompress);
    var newStream = new MemoryStream();
    ds.CopyTo(newStream);
    ds.Flush();
    ds.Close();
    return newStream.GetBuffer();
}

The first block deflates as expected. Subsequent blocks are the right inflated size but, seemingly randomly, some bytes are 0 when they shouldn't be, usually in in groups of 4-12.

How I can deflate different blocks of zipped data while maintaining the same history buffer?

UPDATE: After a little more research it looks like in MSZIP compression these blocks ARE the results of separate deflate operations but the "history buffer" is maintained between them I don't know if deflatestream is gonna be able to handle this. updated the actual question


Solution

  • Yes, there is something that you are missing. Each deflate block can and does use history from the previous deflate block. So at each block you must initialize the deflate dictionary with the last 32K of uncompressed data from the previous blocks.