boto3 copy using SourceClient and access keys results in "AccessDenied"

Question

I am trying to determine if using boto3 copy with a SourceClient will work for my current use case. The documentation mentions the SourceClient parameter "The client to be used for operation that may happen at the source object" . I have two buckets that require key access and I need to transfer files between them.

def copy_function(src_bucket, src_key, src_akid, src_sak, dest_bucket, dest_key, dest_akid, dest_sak):
    src_client = boto3.client(
        's3',
        aws_access_key_id=src_akid,
        aws_secret_access_key=src_sak
    )

    dest_client = boto3.client(
        's3',
        aws_access_key_id=dest_akid,
        aws_secret_access_key=dest_sak
    )

    rsp = dest_client.copy(
        CopySource={
            'Bucket': src_bucket,
            'Key': src_key
        },
        Bucket=dest_bucket,
        Key=dest_key,
        SourceClient=src_client
    )

But this results in the following error:

{
  "errorMessage": "An error occurred (AccessDenied) when calling the UploadPartCopy operation: Access Denied",
  "errorType": "ClientError",
  ...
  "stackTrace": [
    "  File \"/var/task/task/lambda_function.py\", line 16, in handler\n    results = test_main(event, context)\n",
    "  File \"/var/task/task/test.py\", line 39, in test_main\n    rsp = dest_client.copy(\n",
    "  File \"/var/task/boto3/s3/inject.py\", line 450, in copy\n    return future.result()\n",
    "  File \"/var/task/s3transfer/futures.py\", line 103, in result\n    return self._coordinator.result()\n",
    "  File \"/var/task/s3transfer/futures.py\", line 266, in result\n    raise self._exception\n",
    "  File \"/var/task/s3transfer/tasks.py\", line 139, in __call__\n    return self._execute_main(kwargs)\n",
    "  File \"/var/task/s3transfer/tasks.py\", line 162, in _execute_main\n    return_value = self._main(**kwargs)\n",
    "  File \"/var/task/s3transfer/copies.py\", line 370, in _main\n    response = client.upload_part_copy(\n",
    "  File \"/var/task/botocore/client.py\", line 565, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n",
    "  File \"/var/task/botocore/client.py\", line 1021, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"
  ]
}

Which seems to suggest the destination client doesn't have permissions to upload but I know the keys provided allow for this. The following works without producing an error:

def copy_function_2(src_bucket, src_key, src_akid, src_sak, dest_bucket, dest_key, dest_akid, dest_sak):
    src_client = boto3.client(
        's3',
        aws_access_key_id=src_akid,
        aws_secret_access_key=src_sak
    )

    rsp = src_client.get_object(
        Bucket=src_bucket,
        Key=src_key
    )

    dest_client = boto3.client(
        's3',
        aws_access_key_id=dest_akid,
        aws_secret_access_key=dest_sak
    )

    rsp = dest_client.put_object(
        Bucket=dest_bucket,
        Key=dest_key,
        Body=rsp.get('Body').read()
    )

    print(rsp)

Have I missed something or does the SourceClient in combination with a destination client not work in the manner I am attempting?

Answer 1

After more digging I discovered a GitHub issue that the credentials used in the main client needs to have permissions in both accounts. The SourceClient is only used for listing and getting information about objects.