Search code examples
javascriptnode.jsamazon-s3aws-sdk-js

How to filter S3 listObject up to a determined point? (Opposite of Marker param)


I am using AWS SDK function listObject() to retrieve objects from a given bucket. The objects have a table name and timestamp in their path, so in order to filter the response, I am using Prefix and Marker:

However, Marker determines which object to start with, I need the opposite: To tell the query where to stop.

s3.listObject({ Bucket: bucket, Prefix: table-name, Marker: `${table-name}/${timestamp}` });

The structure of the bucket is as follows:

table-name/timestamp/uid

I want to get all objects up to but not including the timestamp.

Edit: I could do this myself using some javascript but I would like let the query do the work for me.


Solution

  • AWS doesn't seem to have native support for this, which makes sense given how Marker & ContinuationToken work, unless AWS adds this, future people with this problem will have to JS-it:

    s3.listObjectV2({ Bucket: bucket, Prefix: table-name }).promise()
    .then(response => {
        return response.Contents.map(item => item.Key)
        .filter(obj => {
              let objTimestamp = obj.match(/\/(.*)\//)[1];
              return new Date(timestamp) > new Date(objTimestamp);
            });
        });
    });