Search code examples
pythonzipcompressionzlibpython-zipfile

Python - faster library to compress data than ZipFile


I am developing a web application in Python 3.9 using django 3.0.7 as a framework. In Python I am creating a function that can convert a dictionary to a json and then to a zip file using the ZipFile library. Currently this is the code in use:

def zip_dict(data: dict) -> bytes:
    with io.BytesIO() as archive:
        unzipped = bytes(json.dumps(data), "utf-8")
        with zipfile.ZipFile(archive, mode="w", compression=zipfile.ZIP_DEFLATED) as zipFile:
            zipFile.writestr(zinfo_or_arcname="data", data=unzipped)
        return archive.getvalue()

Then I save the zip in the Azure Blob Storage. It works but the problem is that this function is a bit slow for my purposes. I tried to use the zlib library but the performance doesn't change and, moreover, the created zip file seems to be corrupt (I can't even open it manually with WinRAR). Is there any other library to increase the compression speed (without touching the compression ratio)?


Solution

  • Try adding a compresslevel=3 or compresslevel=1 parameter to the zipfile.ZipFile() object creation, and see if that provides a more satisfactory speed and sufficient compression.