Search code examples
powershellamazon-web-servicesamazon-s3aws-powershell

Error in PowerShell due to copying the content of an S3 bucket


I copy the content of an S3 bucket to a local directory, however I get an error output from the powershell.

Copy-S3Object : The requested range is not satisfiable

It is pointing to this command:

Copy-S3Object -BucketName $bucket -Key $object.Key -LocalFile $localFilePath -Region $region

Why do I get this error ? Note that the desired files that are needed to be copied do indeed get copied locally.


Solution

  • I can't say why you are getting that error returned from S3, but I can tell you that if you are copying multiple objects you probably want to use the -LocalFolder parameter, not -LocalFile. -LocalFolder will preserve the prefixes as subpaths.

    When downloading one or more objects from S3, the Read-S3Object cmdlet works the same as Copy-S3Object, but uses -KeyPrefix to specify the common prefix the objects share, and -Folder to indicate the folder they should be downloaded to.

    This also reminds me I need to check why we used -LocalFolder on Copy-, and -Folder on Read- although I suspect aliases may also be available to make them consistent.

    HTH

    (Edit): I spent some time this morning reviewing the cmdlet code and it doesn't appear to me the cmdlet would work as-is on a multi-object download, even though it has a -LocalFolder parameter. If you have a single object to download, then using -Key/-LocalFile is the correct parameter combination. If -LocalFolder is passed, the cmdlet sets up internally to do a single file download instead of treating -Key as a common key prefix to a set of objects. So, I think we have a bug here that I'm looking into.

    In the meantime, I would use Read-S3Object to do your downloads. It supports both single (-Key) or multi-object download (-KeyPrefix) modes. https://docs.aws.amazon.com/powershell/latest/reference/index.html?page=Read-S3Object.html&tocid=Read-S3Object