Search code examples
amazon-web-servicesamazon-s3aws-sdk

Is it safe to fill CompleteMultipartUpload() parts list with ListParts() result?


on CompleteMultipartUpload API we have to put the list of uploaded parts E-tags.

I saw that ListParts result is not the same as my app state about parts. all of my UploadPart calls don't have any error but Sometimes the ListParts misses some of them.

So, is it right to fill these E-tags by the result of ListParts API result? If not, why the results are not the same?


Solution

  • It is not safe to use the ListParts response data when calling CompleteMultipartUpload. Doing so can mask problems in your code.

    Only use the returned listing for verification. You should not use the result of this listing when sending a complete multipart upload request. Instead, maintain your own list of the part numbers you specified when uploading parts and the corresponding ETag values that Amazon S3 returns.

    https://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html

    ListParts is only intended for sanity checking, for comparison. You need to use the etags you received when uploading the original parts. If the ListParts response is incomplete, that may only be an eventual consistency issue in S3 and a subsequent poll may return the apparently-missing parts. If the ListParts response is actually in conflict with what you received when uploading the parts, there is a problem with your code or corruption on the wire or (unlikely) inside S3.

    A multipart upload client has no need for the ListParts action.