How to zip files in memory and send to Amazon S3

Question

I am having this problem that if I write something like (where bunch is just a list of file paths):

zip_buffer = io.BytesIO()
with zipfile.ZipFile(zip_buffer, "a", zipfile.ZIP_DEFLATED, False) as zipper:
        for file in bunch:
            zipper.write(file)
with open('0.zip', 'wb') as f:
    f.write(zip_buffer.getvalue())

Then I get a zip file 0.zip, with all the files in the "bunch" list. Thats great.

However when I try to upload to Amazon S3 from memory

s3_client = boto3.client("s3")
zip_buffer = io.BytesIO()
with zipfile.ZipFile(zip_buffer, "a", zipfile.ZIP_DEFLATED, False) as zipper:
     for file in bunch:
         zipper.write(file)
s3_client.put_object(Bucket=S3_BUCKET, Key=ZIP_NAME_IN_S3_BUCKET, Body=zip_buffer.getvalue())

Then the zip file is created in the Amazon S3 bucket, however its not a valid zip file that can be unzipped. Why is it different when I save it locally, and send it to Amazon S3?

Answer 1

I found the solution. I had to do:

s3_client = boto3.client("s3")
zip_buffer = io.BytesIO()
with zipfile.ZipFile(zip_buffer, "a", zipfile.ZIP_DEFLATED, False) as zipper:
     for file in bunch:
         zipper.write(file)
zip_buffer.seek(0)
s3_client.upload_fileobj(zip_buffer, S3_BUCKET, ZIP_NAME_IN_S3_BUCKET)