Search code examples
amazon-web-servicesaws-sdkaws-cliaws-sdk-js

Does AWS SDK or CLI provide offset support on pagination when listing S3 bucket?


Does AWS offer any kind of offset when paginating objects inside S3 bucket, either using SDK or CLI? As I am implementing regular navigation in my client, I need to have support for offset pagination when user clicks on the wanted page number.

The StartAfter can be used only to retrieve next x results from the defined Key. Also retrieving all objects from the bucket and then doing the pagination on the server breaks the pagination principle itself (it doesn't make sense).

The language I'm using is Javascript through the JS SDK, but it doesn't matter if a CLI command can be used for this.


Solution

  • The ListObjects() API call will return a maximum of 1000 objects.

    If more objects are available, isTruncated will be True in the response and NextContinuationToken will contain a token that can be passed via ContinuationToken in a subsequent call. This will return the next set of results.

    However, it is not possible to directly jump to a specific page of results (eg going from the first response of 1000 objects to the Nth set of results). Your code will need to sequentially request the next set until the Nth set of results is received.

    Thus, rather than thinking of the API call as providing 'pagination', think of it as providing 'continuation' for a result set of unknown length.

    If you are using the AWS CLI, it automatically implements the 'continuation' for you when using the aws s3 ls s3://bucket-name command.