Search code examples
amazon-web-servicesaws-java-sdk

aws s3 java sdk list objects

I have a structure as so:

common/test/v20170522/part-0001.snappy
common/test/v20170522/part-0002.snappy
common/test/v20170522/part-0003.snappy
common/test/v20170522/part-0004.snappy
common/test/v20170622/part-0001.snappy
common/test/v20170622/part-0002.snappy
common/test/v20170622/part-0003.snappy
common/test/v20170622/part-0004.snappy

There are a lot more version folders with a lot more files in them. I basically want a listing of all of the folders in the test/common folder: so v20170522, v20170622, etc.

Here is what I have so far:

ListObjectsRequest listObjectsRequest = new ListObjectsRequest();
listObjectsRequest.setBucketName("common");
listObjectsRequest.setPrefix("test/");
listObjectsRequest.setDelimiter("/");

This returns to me just the files that are in the common/test folder and not the folders. When I remove the delimiter, I get all of the subfiles of all the folders. Is there a way of doing this?

Solution

  • It's important to know that S3 is not a regular file system and that the list operation works a little differently to what you might expect.

    When invoking listObjects, you specify:

    • prefix which limits results to only those keys that begin with the specified prefix
    • delimiter which causes listObjects to roll up all keys that share a common prefix into a single summary list result

    You should find the 'folders' in the returned common prefixes, accessible via getCommonPrefixes().

    For more, see Listing Keys using Prefix and Delimiter.