I have CloudFront serving my Angular pages and Elastic Beanstalk running a nodejs server that handles the API calls.
One call in particular takes a lot of time. It reads files that are stored in a flat structure and puts them into a file structure then zips them up and returns them to the user. Most of these files are > 100 MB in size so it takes time. I have put CloudFront's Origin Response Timeout at 60 (which it says is the maximum) and the data I am working with will work with that much time but we are only getting started and I know that there will be times later on when 60 seconds is not going to be long enough.
CloudFront does seem to make 3 attempts (the original plus 2 retries) so it would be a total of 3 minutes but when the first retry comes in then the process will not complete. I even put in a hack that checks to see if the process has already been started in subsequent calls but it still hangs the process and ends up returning a 504.
My question is is there a way to have CloudFront wait longer than 60 seconds or is there something that I can periodically return that says I am still working and reset the timeout? Everything that I have tried so far doesn't work.
Seems the normal trend for me. After searching everywhere for a day or two I post a question and then not long after that I find a solution. Here is what I found:
You can make a request to increase CloudFront's Origin Response Timeout (see here). Not sure why I didn't find this page sooner. Still the problem is that if I increased it and kept it the way I had it that would mean the timeout for all the requests to my api server would be this high value and only this one request needs the higher limit (at least as of now). So, I created a new origin and behavior that only pertains to that one api call and can set the limit higher and leave the limit lower for all the other api calls.
At least that is the theory. My testing bears this out but haven't heard if my request for the limit increase has been granted.