Search code examples
amazon-s3zips3cmd

Corrupted files when downloading large zip file from S3


I have a zip file with the size 1 GB on S3 bucket. After downloading, I can't seem to unzip it. It always says

End-of-central-directory signature not found.  Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.

Later, I download it again, using s3cmd this time. It says

WARNING: MD5 signatures do not match: computed=384c9a702c2730a6b46d21606137265d, received="b42099447c7a1a390d8e7e06a988804b-18"

Is there any s3 limitation I need to know or this is a bug?


Solution

  • Thanks @ergoithz for reminding me that I had this question :)

    The problem is already fixed, with AWS SDK for nodejs being the problem. Apparently it cannot upload large files using stream data fs.createReadStream(), so I switched to using Knox where it worked perfectly