Search code examples
windowszipcompressionarchivezstandard

Decompressing large zst files

I have a 16gb compressed zst file. how would I decompress it on windows? I do not have the memory to handle the decompressed version, so it needs to be written straight to a file.

Solution

  • At the risk of sounding obvious, you need a windows decompressor that supports zstandard.

    At the zstandard releases page, there are two prebuilt Windows executables, zstd-v1.5.5-win32.zip and zstd-v1.5.5-win64.zip available.

    Regarding the amount of memory needed, zstandard will not decompress it completely into memory before writing to the filesystem -- it works in a streaming mode loop that operates like this pseudo-code

    WHILE NOT at eof of zstd file
    read a small part of the compressed data into memory
    uncompressing it into memory
    writes the uncompressed data to disk