AWS S3 find files by name

Question

I'm really suprised that there is no easy way to scan all of files in S3 via aws-sdk node package and find some file by his name or some regex.

Here is an example of folder structure:

- mainDirectory

    - directoryOne
        (...)
        - file1.png
        - file2.png
        (...)

    - directoryTwo
        (...)
        - file11.png
        - file22.png
        ( ...)

    - directoryThree
        (...)
        - file111.png
        - file222.png
        (...)

Is there any option to use Delimiter or Prefix in that way to list files from EVERY directory named file(...)?

Tried this:

const params = {
    Bucket: process.env.AWS_S3_BUCKET,
    Delimiter: '/',
    Prefix: 'file'
};

Answer 1

No, the prefix in list api is used to match object keys starting from the beginning, not relative to part of the key.

If you have a huge amount of objects and timing doesn't matter for you, then I recommend setting up S3 inventory to generate a daily inventory report. Then you can process it using Athena with some SQL queries.

If you need to scan it and get the objects in real time, I would recommend also saving the object key to other data structure for fast query.