Search code examples
google-app-enginedefaulttask-queue

App Engine: What are the Default Retry Parameters for Failed Taskqueues

Google App Engine has a robust set parameters to automatically retry failed tasks in the distributed tasksqueue. While the documentation explains what these parameters do, it fails to specify what the default parameters are.

What are the defaults for the Taskqueue's retry parameters. More specifically, what are the default values for the following retry parameters:

  • task_retry_limit
  • task_age_limit
  • min_backoff_seconds
  • max_backoff_seconds
  • max_doublings
Solution

  • The default behaviour is described at the beginning of this section:

    Tasks executing in the task queue can fail for many reasons. If a task fails to execute (by returning any HTTP status code outside of the range 200–299), App Engine retries the task until it succeeds. By default, the system gradually reduces the retry rate to avoid flooding your application with too many requests, but schedules retry attempts to recur at a maximum of once per hour until the task succeeds.

    From this description and by observing the logs for an actually failing task in a push queue with no retry config (thus using defaults for all these parameters) I "derived" these values:

    • task_retry_limit: None
    • task_age_limit: None
    • min_backoff_seconds: 0.1
    • max_backoff_seconds: 3600
    • max_doublings: None

    These would be some of the logs I looked at on my development server - I didn't wait for the retry interval to settle down, tho:

    INFO     2015-09-02 12:50:54,670 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:50:54,670 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 0.100 seconds
INFO     2015-09-02 12:50:54,774 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:50:54,774 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 0.200 seconds
INFO     2015-09-02 12:50:54,983 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:50:54,983 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 0.400 seconds
INFO     2015-09-02 12:50:55,394 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:50:55,394 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 0.800 seconds
INFO     2015-09-02 12:50:56,206 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:50:56,206 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 1.600 seconds
INFO     2015-09-02 12:50:57,815 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:50:57,815 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 3.200 seconds
INFO     2015-09-02 12:51:01,058 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:51:01,058 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 6.400 seconds
INFO     2015-09-02 12:51:07,507 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:51:07,507 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 12.800 seconds
INFO     2015-09-02 12:51:20,346 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:51:20,346 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 25.600 seconds
INFO     2015-09-02 12:51:45,988 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:51:45,989 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 51.200 seconds
INFO     2015-09-02 12:52:37,224 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:52:37,225 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 102.400 seconds
INFO     2015-09-02 12:54:19,668 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:54:19,668 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 204.800 seconds
INFO     2015-09-02 12:57:44,508 module.py:808] my_module: "POST /my/task_q/path HTTP/1.1" 403 55
WARNING  2015-09-02 12:57:44,509 taskqueue_stub.py:1977] Task task2 failed to execute. This task will retry in 409.600 seconds

    Edit: Actually I found an even better answer which I used to correct mine (I previously missed the 0.100s log entries burried between others): What are the defaults for a task queue in AppEngine?