I've recently setup a new site which utilises Amazon Cloudfront to distribute very large files, however Amazon is currently making so many requests to my server for so long that my entire site is coming to a stand still.
I should note that i'm not using S3, cloudfront is connecting direct to my server.
I have a 100mb data connection and the files i'm trying to distribute are two 3GB files. However if I run iftop in ssh Amazon ip addresses seem to take up every row probably trying to cache the same file to multiple different servers and they appear to be using up my entire connection.
Is there anyway to limit cloudfront to a connection of say 10mb's or less?
Are you sure that you are setting the caching headers properly for your files? CloudFront respects the Expires and Cache-Control headers, which you can use to both extend and reduce the amount of time that a file is considered valid. Adding Cache-Control: public, max-age=86400 to your response headers will cause edge servers to cache your files for up to a day (86400 seconds).
Another thing to note is that edge servers have limited storage capacities for caching files. Given the choice between keeping a 10GB file that is used semi-frequently and a 10KB file that is used less frequently, Amazon may elect to remove the 10GB file in order to serve more customers. If possible, consider reducing your object size to avoid being expunged.