Search code examples
amazon-web-servicesgoamazon-s3aws-sdkaws-sdk-go

Get resource URI on s3 bucket


Currently, I construct URI by hand with string interpolation:

fmt.Sprintf("https://%s.%s.amazonaws.com/%s/%s", serviceId, region, namespace, filename)

which IMHO is bad as I pass them via parameters from env variables. I have access to aws.Session object which is correctly initialized with our s3 bucket.

Is there are a semantically correct way(probably with aws sdk API) to generate the same resource URI I do by dummy string builder?


Solution

  • From an example...

    While aws-sdk-go does expose the *http.Request in a struct returned by svc.GetObjectRequest, the URL it returns is really more of a 'url format' until you run a method on the request like Sign or Presign.

    A few Presign examples can be found On This Page. The Get example provides the following output:

    2017/08/03 09:47:43 The URL is https://get-resource-uri-test.s3-us-west-2.amazonaws.com/myKey?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIFJBK7YB3H7CTZIA%2F20170803%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20170803T144743Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=3c279aece1eda3c33b0711bb0e5a86e330e378f9052f1e73cdb0b9ca19de6209

    Note that I'm using 'get-resource-uri-test' as my bucket name since the one in the example was already in use


    Adjusted example

    Since this isn't the kind of output you're looking for, I've adjusted the example to use Sign, which keeps things more clean since the auth info is put into headers/etc. instead of in the URL:

    svc := s3.New(session.New(&aws.Config{Region: aws.String("us-west-2")}))
    req, _ := svc.GetObjectRequest(&s3.GetObjectInput{
        Bucket: aws.String("get-resource-uri-test"),
        Key:    aws.String("myKey"),
    })
    
    // Instead of using Presign, which was provided in example
    if err := req.Sign(); err != nil {
        log.Println("Failed to sign request", err)
    }
    
    log.Println("The URL is", req.HTTPRequest.URL)
    

    2017/08/03 09:53:22 The URL is https://get-resource-uri-test.s3-us-west-2.amazonaws.com/myKey


    Considerations

    While this suggestion does avoid actually making any calls to S3, I do have mixed feelings about it. In some respects, aws-sdk-go seems designed to somewhat eliminate the need for URL manipulation and instead generates it (mostly internally) as a consequence of what you're trying to accomplish.

    So is it really necessary for us to create a whole S3 client to get a properly 'sanctioned/blessed' url from Amazon in Go? It seems a little overboard, but this is the only way I've found so far to receive a formatted URL from aws-sdk-go. If anyone else is able to come up with a more reasonable alternative, I'd encourage that person to post his/her solution as well.

    I probably don't need to say this, but if this example is something you're planning to use at scale, I'd recommend creating a single s3 client (named svc in the example provided) and reusing its GetObjectRequest method to avoid creating a new client every time you need to determine a URL.