Unzip file in stream request gives UnexpectedSignatureError

Question

Trying to get a file export from a request which is in zip format. I want to download line by line since later I will be uploading to cloud storage and do not want to store the whole file in memory.

I tried the following:

with requests.get(export_url, stream=True) as response:
    for file_name, file_size, unzipped_chunks in stream_unzip(response.iter_lines()):
        for chunk in unzipped_chunks:
            print(chunk)

However I am getting the following error that I am not sure how to solve or what it means.

UnexpectedSignatureError: b'<htm'

Answer 1

Try this:

from stream_unzip import stream_unzip
import httpx

def zipped_chunks():
    # Iterable that yields the bytes of a zip file
    with httpx.stream('GET', 'https://www.example.com/my.zip') as r:
        yield from r.iter_bytes(chunk_size=65536)

for file_name, file_size, unzipped_chunks in stream_unzip(zipped_chunks()):
    for chunk in unzipped_chunks:
        print(chunk)