I need to serve out large files (10-50GB) at low volumes from an S3 bucket. I am using CloudFront because I need Lambda@Edge to inspect the requests before they hit the s3 bucket.
From the AWS docs here I understand that I pay for Data Transfer Out to Internet. I also understand the largest object size for the CloudFront Cache is 20GB.
My question is: for files less than 20GB, does not caching these files have any impact on the cost/pricing of CloudFront? Does setting the Cache-control header to no-cache mean that the response bypasses CloudFront? I assume there must be some costs associated with the caching/storing the file on the edge servers.
There is a similar question here but the answer doesn't discuss cost/pricing of caching specifically.
For objects smaller than 20 GB, whether or not you prevent CloudFront from caching them (typically by setting Cache-Control to any combination of private, no-cache, or no-store while leaving Minimum TTL set to the default value of 0 -- although there are also other ways) would have no impact on pricing, because CloudFront doesn't charge anything for cache storage. While this might be a surprise, remember that responses are only cached in edges through which they are requested, and that CloudFront's cache is a cache, and thus ephemeral, so CloudFront can discard cached objects that don't see frequent traffic.
Preventing caching doesn't bypass CloudFront, since CloudFront is the service that is processing the entire request. It just prevents the response from being stored in the cache as it's being returned to the viewer.
You will want to verify that objects larger than 20 GB will work at all. The documentation suggests they will not.
Maximum File Size
The maximum size of a response body that CloudFront will return to the viewer is 20 GB. This includes chunked transfer responses that don't specify the Content-Length header value.