amazon-web-services amazon-s3 pre-signed-url

How can I add tags to an object when using presigned AWS S3 URLs?

I'm using @aws-sdk/s3-request-presigner on a Node.js server to generate presigned URLs which my web application is then using to upload files to my S3 bucket. This works well, but I am struggling to add tags to the files.

Here is my current server side code:

    const putObjectCommandParams: PutObjectCommandInput = {
      Bucket: process.env.AWS_S3_BUCKET_NAME,
      Key: fileKey,
      ContentType: contentType,
      Tagging: "org=abc&y=2021"
    };
  
    const command = new PutObjectCommand(putObjectCommandParams);
    const signedUrl = await getSignedUrl(s3Client, command, { expiresIn: 3600 });

Here is my current web application code:

    const xhr = new XMLHttpRequest();
    xhr.open('PUT', signedUploadUrl);
    xhr.setRequestHeader('Content-Type', contentType);
    xhr.onload = () => {
      if (xhr.status === 200) {
        alert('File uploaded successfully.');
        this.fileUploaded(fileName, presignedResponse.file);
      } else {
        alert('Could not upload file. Status: ' + xhr.status);
      }
    };
    xhr.onerror = (err) => {
      alert('Could not upload file. ' + err);
    };
    xhr.send(file);

The file uploads successfully and appears in S3, but the tags are blank when I check in the AWS console.

As some other answers suggest, I've tried to add the exact same tag string to an "X-Amz-Tagging" header in the file upload request, but then the upload fails and I then get the following error:

<Error>
<Code>AccessDenied</Code>
<Message>There were headers present in the request which were not signed</Message>
<HeadersNotSigned>x-amz-tagging</HeadersNotSigned>
...

Does anyone know why this might be or can you see anything obvious I am doing wrong? Thank you!

Solution

x-amz-tagging also needs to be signed & sent in the request header, unlike other headers that can be a query string.

Taking a look at the source code, @aws-sdk/s3-request-presigner will by default 'hoist' all headers - including x-amz-tagging - to the query parameters of the presigned URL. An exception is x-amz-server-side-encryption but x-amz-tagging should also be added.

As a result of the hoisting, S3 doesn't add the tags to the object.

With the current code, and its accompanying SDK implementation, you'll get something similar to:

https://xxx.s3.eu-west-1.amazonaws.com/xxx.xxx?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=xxxx&X-Amz-Date=xxx&X-Amz-Expires=3600&X-Amz-Signature=xxx
&X-Amz-SignedHeaders=host
&x-amz-tagging=org%3Dabc%26y%3D2021
&x-id=PutObject

Note that x-amz-tagging is not in the value for X-Amz-SignedHeaders.

Also, only adding it manually to the upload request won't work, as the original signing request didn't include the header to be signed. This is why S3 returns the 'There were headers present in the request which were not signed' error. In other words, 'You're sending the x-amz-tagging header when using the URL but you didn't originally sign it when you created the URL'.

There are 2 changes that need to be made:

Change	Description	Why?
1	Configure `s3-request-presigner` to not hoist `x-amz-tagging`, when creating the URL.	To ensure it is a signed header for S3 as it is a prerequisite for S3 accepting any request with the `x-amz-tagging` header provided.
2	Send the same tag keys & values (`Tagging`) as the value for the `x-amz-tagging` header when using the URL (to upload).	To actually pass the tags to S3 to store against the object.

Change 1 can be done by adding x-amz-tagging to the unhoistableHeaders field of the parameter object we pass to getSingedUrl:

const signedUrl = await getSignedUrl(s3Client, command, {
    expiresIn: 3600,
    unhoistableHeaders: new Set(['x-amz-tagging']),
});

Your pre-signed URLs will then start to look like:

https://xxx.s3.eu-west-1.amazonaws.com/xxx.xxx?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=xxxx&X-Amz-Date=xxx&X-Amz-Expires=3600&X-Amz-Signature=xxx
&X-Amz-SignedHeaders=host;x-amz-tagging
&x-id=PutObject

Note that x-amz-tagging is now within the value for X-Amz-SignedHeaders, and that it is no longer sent as a query parameter via &x-amz-tagging.

Change 2 can be done by including the x-amz-tagging header when making the XHR request:

const objectTags = 'org=abc&y=2021';

const xhr = new XMLHttpRequest();
xhr.open('PUT', signedUploadUrl);
xhr.setRequestHeader('Content-Type', contentType);
xhr.setRequestHeader('x-amz-tagging', objectTags);
...

With these changes, the x-amz-tagging header will be signed in the URL & also included in the headers of the request, enabling S3 to add the tags to the object.

Here is a complete yet minimal working Node.js CLI app to demonstrate the above:

// package.json

{
  "name": "aws-sdk-js-presigned-url-tagging",
  "version": "1.0.0",
  "dependencies": {
    "@aws-sdk/client-s3": "^3.441.0",
    "@aws-sdk/s3-request-presigner": "^3.441.0"
  }
}

// main.js

const {
    S3Client,
    PutObjectCommand,
    GetObjectTaggingCommand
} = require('@aws-sdk/client-s3');
const {getSignedUrl} = require('@aws-sdk/s3-request-presigner');
const {writeFile, readFile} = require('fs/promises');

const s3Client = new S3Client({region: 'eu-west-1'});

const bucketName = 'xxx';
const fileKey = 'xxx.xxx';
const contentType = 'application/octet-stream';
const objectTags = "org=abc&y=2021";

async function createLocalFile() {
    await writeFile(fileKey, 'abc', 'utf8');
    console.log('Local file created...');
}

async function createPresignedUrl() {
    const putObjectCommandParams = {
        Bucket: bucketName,
        Key: fileKey,
        ContentType: contentType,
        Tagging: objectTags,
    };

    const presignedUrlParams = {
        expiresIn: 3600,
        unhoistableHeaders: new Set(['x-amz-tagging']),
    };

    const putObjectCommand = new PutObjectCommand(putObjectCommandParams);
    const signedUrl = await getSignedUrl(s3Client, putObjectCommand, presignedUrlParams);
    console.log('Presigned URL:', signedUrl);

    return signedUrl;
}

async function uploadFile(signedUrl) {
    const fileData = await readFile(fileKey);

    await fetch(signedUrl, {
        method: 'PUT',
        body: fileData,
        headers: {
            'Content-Type': contentType,
            'x-amz-tagging': objectTags
        },
    });

    console.log('File uploaded successfully...');
}

async function getObjectTags() {
    const getObjectTaggingCommandParams = {
        Bucket: bucketName,
        Key: fileKey
    };
    const command = new GetObjectTaggingCommand(getObjectTaggingCommandParams);
    const {TagSet} = await s3Client.send(command);

    console.log('Object tags after uploading object:', TagSet);
}

async function doStuff() {
    await createLocalFile();
    const signedUrl = await createPresignedUrl();
    await uploadFile(signedUrl);
    await getObjectTags();
}

doStuff();

Output:

Local file created...

Presigned URL: https://xxx.s3.eu-west-1.amazonaws.com/xxx.xxx?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=xxx&X-Amz-Date=xxx&X-Amz-Expires=3600&X-Amz-Signature=xxx&X-Amz-SignedHeaders=host%3Bx-amz-tagging&x-id=PutObject

File uploaded successfully...

Object tags after uploading object: [ { Key: 'org', Value: 'abc' }, { Key: 'y', Value: '2021' } ]