I've set up a TaskQueue for my AppEngine API. The API processes large requests, and may take up to three hours to conclude computing, working on one task at a time.
I've set 'max_concurrent_requests' to 1 in the queue.yaml file, so that only one task will be active at a time, and increased the gunicorn timeout as well.
My problem comes because the CloudTask requests seem to have a timeout of 10min, after which they throw up an error: Process terminated because the request deadline was exceeded. Please ensure that your HTTP server is listening for requests on 0.0.0.0 and on the port defined by the PORT environment variable. (Error code 123)
How can I configure my queue to wait idle for a previous task to finish, instead of simply timing out?
You've most likely set your App Engine service scaling element to "autoscaling" (or didn't define it, autoscaling being the default value) in the app.yaml file.
Instances in autoscaling have a deadline of 10min, as documented here. You'll need to re-deploy your service with an updated app.yaml file setting the scaling element to "manual scaling" or "basic scaling" to allow your tasks to run to up to 24h.