S3 direct upload using a Pre-signed url generated from STS temporary credentials

Question

I am trying to upload files to s3 using a pre-signed URL. I could successfully generate the URL using the AssumeRole credential but while uploading the file from Browser it is throwing the below error.

<Code>InvalidAccessKeyId</Code>
    <Message>The AWS Access Key Id you provided does not exist in our records.</Message>

Method1:

Code used to generate presigned URL

cred = Aws::AssumeRoleCredentials.new(role_arn: ENV['AWS_ROLE_ARN'], role_session_name: 'xxx_service').credentials
post = Aws::S3::PresignedPost.new(creds, S3_REGION, S3_BUCKET, {
        key: Rails.env + '/' + file_name,
        metadata: {
          'original-filename' => file_name
        },
        acl: 'private',
      })

 url =  post.url,
 fields =  post.fields

Frontend Code to upload file from Browser

var form = new FormData();
form.append("key", "demo/test.xxx");
form.append("x-amz-meta-original-filename", "test.xxx");
form.append("policy", "XXXXXX");
form.append("x-amz-credential", "AXXXXXXX");
form.append("x-amz-algorithm", "AWS4-HMAC-SHA256");
form.append("x-amz-date", "20230530T145924Z");
form.append("x-amz-signature", "4XXXXXXXXXXXXXXX");
form.append("file", fileInput.files[0], "test.xxx");
form.append("x-amz-security-token", "IQXXXXXXXXXXX");

var settings = {
  "url": "https://bucket-url",
  "method": "POST",
  "timeout": 0,
  "processData": false,
  "mimeType": "multipart/form-data",
  "contentType": false,
  "data": form
};

$.ajax(settings).done(function (response) {
  console.log(response);
});

The same code was working using the IAM user credential. Due to CyberSecurity requirements, we have started using the AssumeRole credential. Even I tried adding x-amz-session-token in the request body but no difference in the output.

Later I found one more way to generate the pre-signed URL which is allowing me to upload file successfully from Brower using the PUT method.

Method2:

signer = Aws::S3::Presigner.new(credentails: creds, region: S3_REGION)
url, _ = signer.presigned_request(
            :put_object, bucket: S3_BUCKET, key: "#{Rails.env}/#{file_name}"
          )

Frontend Code to upload file from Browser

var form = new FormData();
form.append("file", fileInput.files[0], "test.xxx");

var settings = {
  "url": "https://bucket-url/demo/Sunspot.xxx?X-Amz-Algorithm=XXXXX&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Credential=XXXXX&X-Amz-Date=20230530T144029Z&X-Amz-Security-Token=XXXXX%3",
  "method": "PUT",
  "timeout": 0,
  "processData": false,
  "mimeType": "multipart/form-data",
  "contentType": false,
  "data": form
};

$.ajax(settings).done(function (response) {
  console.log(response);
});

The only problem with the above solution is I am not able to reparse file .xxx after downloading it from S3. I have tested this solution by uploading PDF and PNG images, it opens properly after downloading it. Maybe S3 modifies the signature/hash of the file .xxx in the second method due to which I am not able to reparse the file. Using the first method and IAM credential, it is working fine.

Is there any other way to upload files directly to S3 from Browser without using IAM user credentials?

Answer 1

I could upload the file using the Presigned POST mechanism itself. Just changing the order of the form data in the frontend code worked for me. Aws error message was misleading, instead of asking us to send the security fields in the correct order it is telling us AccessKEY is not found. I just interchanged the position of x-amz-signature and x-amz-security-token

Working code

var form = new FormData();
form.append("key", "demo/test.xxx");
form.append("x-amz-meta-original-filename", "test.xxx");
form.append("policy", "XXXXXX");
form.append("x-amz-credential", "AXXXXXXX");
form.append("x-amz-algorithm", "AWS4-HMAC-SHA256");
form.append("x-amz-date", "20230530T145924Z");
form.append("x-amz-signature", "4XXXXXXXXXXXXXXX");
form.append("file", fileInput.files[0], "test.xxx");
form.append("x-amz-security-token", "IQXXXXXXXXXXX");

var settings = {
  "url": "https://bucket-url",
  "method": "POST",
  "timeout": 0,
  "processData": false,
  "mimeType": "multipart/form-data",
  "contentType": false,
  "data": form
};

$.ajax(settings).done(function (response) {
  console.log(response);
}); 

We need to send security fields in the correct order while uploading files to S3.

• key
• x-amz-algorithm
• x-amz-credential
• x-amz-date
• policy
• x-amz-security-token
• x-amz-signature

I found this from here