Copying an s3 bucket to another bucket is too slow, how can I do it faster?

Question

Copying an S3 bucket to another bucket is too slow using the following command:

aws s3 cp s3://bucket1 s3://bucket2 --recursive

Using awscli. But this is too slow. Because my bucket1 contains too many files. Only 3,000,000 images copy in 12 hours.

Looking for recommendations to help expedite the copy process.

Answer 1

You can use this script with multi-threads:

https://github.com/paultuckey/s3_bucket_to_bucket_copy_py