Search code examples
amazon-web-servicescachingamazon-s3cloudinaryfastly

How can I stagger purging multiple cascading CDN's to assure a complete purge?

I use an Amazon S3 bucket, a cloudinary cache, and a fastly cache. In conjunction, they deliver images of any shape, size, or other transformation you ask for, and very fast. However, they propagate purge requests at different rates.

Here is the cascading arrangement:

  • When an image is requested, Fastly tries to serve that image from it's cache.
  • If the image is absent, Fastly asks Cloudinary for that image.
  • Cloudinary tries to serve the image serve from its cache.
  • If the image is absent, Cloudinary checks to see if the requested image had associated transformation parameters.
  • After finding the transformation params, Cloudinary tries to find an untransformed version of the image in its cache to apply the transformation to.
  • If the untransformed image is absent, Cloudinary will request it from the S3 bucket.
  • Cloudinary then applies the transformation and caches both the original and the transformed versions.
  • Cloudinary serves the transformed image to Fastly.
  • Fastly caches the transformed image and serves it.

I'd like to completely remove an image and all transformed versions (derivatives) of that image from all of my services. Cloudinary takes an hour to propagate the DELETE request to all of its servers.

I see that it is best to delete first in S3, then Cloudinary, and finally to purge fastly. How best does one delay a purge call for an hour though? What is the best practice, programmatically speaking, in this situation?

Solution

  • As part of its solution, Cloudinary provides CDN services via Akamai (integration to other CDNs is also available).

    Images are cached on the CDN after initial delivery and until a purge is requested. The propagation to all CDN nodes takes time and may last up to 1 hour, however usually only takes several minutes.

    Coordinating between two CDN layers is a quite complex task, therefore the best practice is to avoid using your own CDN in front of Cloudinary's, especially if invalidations are commonly required.