Search code examples
proxyamazon-vpcamazon-ecscaching-proxy

Caching proxy solutions for internet bound HTTPS traffic

Sorry if this is inappropriate for SO but wasn't sure where best to ask this question!

Background:

  • Running applications on EC2 Container Service (ECS) inside an AWS VPC.
  • There's the potential to move the functions making the requests to Lambda functions in the near future (3-6 months).

What I'm functionally looking to achieve:

  • Cache responses from HTTPS traffic to specific URL patterns (eg. subdomain.example.com) for specified periods (eg. 7 days).
    • We're hitting API limits for free/paid services and looking to inject a layer to handle duplicate requests transparently, not easy to handle at the application layer unfortunately.
  • Have this applied at a VPC (eg. InternetGateway?) level or ECS service level - not too fussed which one.

Ideally this is transparent to the application itself that'd be fantastic but guessing the fact it's HTTPS traffic may throw a spanner in the works of that. Was initially thinking this may be possible at the InternetGateway level but assuming that doesn't have easy access to request headers.

Potential solutions:

Any suggestions would be greatly appreciated!

Alex

PS. As you can probably tell I'm a little out of my depth in this one, sorry if I'm mixing patterns/solutions!

Solution

  • If I understand your question correctly you want to cache certain responses that you do towards paid/free API's of 3rd parties. I'm wondering wether you're looking for a solution that works inside your VPC or if it's fine if the solution is outside.

    When you're OK with some solution running outside of your VPC, Cloudfront might be something worth looking into. Cloudfront can act as a caching layer for any content of any origin, even if the origin connection is using HTTPS. It is even possible to use signed URL's or signed cookies with Cloudfront to restrict unwanted access, if that's what you're going for.