Search code examples
angularamazon-s3.net-corepre-signed-urlminio

Usage of presigned urls in a (e.g. Angular) SPA blows up browser cache

In our project we decided to use presigned urls as basic authentication mechanism.

Trimmed down our setup involves

  • the storage server
  • the api server
  • the client (angular SPA running in the browser)

We use presigned urls for uploading and downloading files from the client directly to the storage server.

Upload flow (simplified):

  • client sends the api: hey I want to upload that
  • api does authorization and validation, does some database stuff and returns a presigned url
  • client uploads directly to the storage server

So far so good. The big problem is the "download" flow.

  • client asks the api: hey, show me a list of what you have
  • api does authorization, validation and returns a json list of objects which also hold the presigned get urls for showing the files (images)
  • the client displays the list of object data and embeds the images downloaded directly from the storage server using the presigned urls

This works great but blows up the browser cache up to multiple GB of RAM.

This happens because the presigned urls generated on mulitple calls are not the same and differ at the authorization part (e.g. holding a new fresh lifetime) on each request. When a user clicks forward and backwards through the paginated list the client will receive different urls and the browser cache treats them as different images.

So far this seems to be a correct behaviour on the browser side (different url equals different image).

So far this seems to be a correct behaviour on the api side (new call will return a new lifetime).

Are there any intended ways how to handle this?

Are the flows themself wrong?

Any ways to solve this beside implementing a centralized presigned url cache when running multiple instances of the api?

May someone could also give advice for meaningful tags I could use.

Solution

  • Every request for a pre-signed resource in your current flow has the browser/client make a new request to S3.

    Hence, the benefit of the browser cache isn't enjoyed and can be done without by specifying additional response headers to control caching policy in the client when generating presigned URLs. The Cache-Control response header can be set in the response headers for the pre-signed request to no-cache. 1

    A better flow I'll suggest is to have the pre-signed URLs have an expiry time between 5 to 15 mins, and set Cache-Control in the response headers of the response of the pre-signed URL to max-age:<expire-time-in-secs>.

    With this new flow, you'll need to ensure that the API server only returns a fresh list of pre-signed URLS after the expiry time by keeping a server side cache as well. You could make gains on the response times from the API server and also avoid serving unnecessary requests from the browser for a cached resource.