Search code examples
javascriptamazon-web-servicesamazon-s3aws-sdk-js

AWS S3 Javascript SDK v2 allows for full path but v3 does not. How can I access the files in S3 with a full path?


My application used to use AWS JavaScript SDK v2. In this version I could have a bucket name with slashes. The pattern of this bucket name is: <bucket_name>//. Essentially mocking a folder structure.

The bucket name I was using is as follows:

bucket-name/folder1/folder2

I realize that S3 does not have a concept of folders. I could pass a string to the getObject method with slashes into the bucket name field. Bucket names cannot have slashes. However, the following code works

const aws = require('aws-sdk');
const s3Client = new aws.S3();
const bucketName = 'bucket-name/folder1/folder2';
const fileName = 'file.txt';
const params = {
   Bucket: bucketName,
   Key: fileName
}

s3.getObject(getParams, function(err, data) {
    // Handle any error and exit
    if (err)
        return err;

  // No error happened
  // Convert Body from a Buffer to a String
  let objectData = data.Body.toString('utf-8'); // Use the encoding necessary
});

I expect that the bucket name would be bucket-name and the Key to be folder1/folder2/file.txt. When I upgrade to v3, this code no longer works and I have to split the path to get the bucket name and the file key.

Here's the v3 code:

import {
    GetObjectCommand,
    S3Client,
} from '@aws-sdk/client-s3';
const bucketName = 'bucket-name/folder1/folder2';
const fileName = 'file.txt';
const s3Client = new S3Client(config);

const fileData = await s3Client.send(
            new GetObjectCommand({
                Bucket: bucketName,
                Key: fileName,
            }),
        );

The above code throws the following exception:

InvalidBucketName: Bucket name shouldn't contain '/', received bucket-name/folder1/folder2

I would expect this error to appear in both v2 and v3 but for some reason the combination of the bucket name and filename I'm passing to the v2 sdk resolves but in the v3 it does not.

Does anyone know why this is happening? Is there a way to pass the above bucket name and filename so that I don't have to separate the bucket name and the file key? For now, I'm doing the following in the v3 code to take in the bucket name with slashes:

const bucketName = 'bucket-name/folder1/folder2';
const fileName = 'file.txt';
const bucketParts = bucketName.split('/');
const finalBucketName = bucketParts[0];
const finalFileName = bucketParts.slice(1).join('/') + '/' + fileName;

Solution

  • The bucket name and object key should be supplied independently as you've noted in the working v3 example:

    const Bucket = 'bucket-name';
    const Key = 'folder1/folder2/file.txt';
    

    My best guess as to why the v2 version worked OK even though you supplied technically invalid bucket and key values is that the v2 SDK simply concatenated these values to form an S3 URI but did not validate them in any way individually. The v3 SDK code seems to be a little more precise and rejects what is technically an invalid bucket name.