Search code examples
amazon-web-servicesamazon-s3multipartaws-cli

AWS S3 CLI multipart upload

When trying to upload a file to an AWS S3 bucket using the aws-cli, does it automatically use multipart upload on bigger files? Because all it outputs is

Completed 3.0 GiB/9.4 GiB (18.4 MiB/s) with 1 file(s) remaining

which does not make me any wiser.

I checked several other information sources and this this page tells me it goes automatically, while the Amazon help tells me otherwise.

Solution

  • They do mention in AWS docs that the CLI commands automatically perform a multipart upload for large objects.

    Excerpt from documentation:

    All high-level commands that involve uploading objects into an Amazon S3 bucket (aws s3 cp, aws s3 mv, and aws s3 sync) automatically perform a multipart upload when the object is large.

    Failed uploads cannot be resumed when using these commands. If the multipart upload fails due to a timeout or is manually cancelled by pressing CTRL+C, the AWS CLI cleans up any files created and aborts the upload. This process can take several minutes.

    If the process is interrupted by a kill command or system failure, the in-progress multipart upload remains in Amazon S3 and must be cleaned up manually in the AWS Management Console or with the s3api abort-multipart-upload command.