How to create a virtual-hosted-style presigned URL for an S3 bucket in a non US region?

Question

I was trying to generate a presigned URL for an S3 bucket in the me-central-1 region using the boto3 library, as demonstrated below

client = boto3.client("s3", region_name="me-central-1")

return client.generate_presigned_url(
    ClientMethod="put_object",
    Params={"Bucket": bucket, "Key": path},
)

This generates a url with the following form: https://bucket.s3.amazonaws.com/path?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential={CREDENTIAL}/{DATE}/me-central-1/s3/aws4_request&X-Amz-Date={DATETIME}&X-Amz-Expires=1800&X-Amz-SignedHeaders=host&X-Amz-Signature={SIGNATURE}

However, trying to use this url results in an IllegalLocationConstraintException with the following message: The me-central-1 location constraint is incompatible for the region specific endpoint this request was sent to.

I did a bit of research into why this was happening and found that S3 endpoints going to regions other than us-east-1 needed to include the region like so: https://bucket.s3.REGION.amazonaws.com/....

I then changed the boto3 client instantation to the version below:

client = boto3.client("s3", region_name="me-central-1", endpoint_url="https://s3.me-central-1.amazonaws.com")

The resulting url had the form: https://s3.me-central-1.amazonaws.com/bucket/path?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential={CREDENTIAL}/{DATE}/me-central-1/s3/aws4_request&X-Amz-Date={DATETIME}&X-Amz-Expires=1800&X-Amz-SignedHeaders=host&X-Amz-Signature={SIGNATURE}

This URL seems to work without returning an IllegalLocationConstraintException! However, I noticed this was now a path-style request instead of a virtual-hosted-style request (as defined here). The provided webpage also states path-style requests will be discontinued, so I will not be able to use the above code for a long-term solution.

My attempts to manually change the path-style request into a virtual-hosted-style request resulted in a SignatureDoesNotMatch error, with the message: The request signature we calculated does not match the signature you provided. Check your key and signing method.

My question would be - is there a way to make the boto3 client return a valid virtual-hosted-style presigned url?

Answer 1

import boto3
from botocore.client import Config

client = boto3.client(
    "s3",
    region_name="me-central-1",
    endpoint_url="https://s3.me-central-1.amazonaws.com",
    config=Config(s3={'addressing_style': 'virtual'})
)

return client.generate_presigned_url(
    ClientMethod="put_object",
    Params={"Bucket": bucket, "Key": path},
)