Google Cloud Tasks enforcing rate limit on forwarding to Cloud Functions?

Question

Cloud Tasks is saying:

App Engine is enforcing a processing rate lower than the maximum rate for this queue either because your application is returning HTTP 503 codes or because currently there is no instance available to execute a request.

However, I am forwarding the tasks to a cloud function using an HTTP POST request, similar to the one outlined in this tutorial. I don't see any 503s in my logs for the cloud function that it forwards to.

My queue.yaml is:

queue:
- name: task-queue-1
  rate: 3/s
  bucket_size: 500
  max_concurrent_requests: 100
  retry_parameters:
    task_retry_limit: 1
    min_backoff_seconds: 120
    task_age_limit: 7d

Answer 1

The problem seems to come from any exception, even though only 503 is listed. If the cloud function responds with any error the task queue slows down the rate, and you have no control over that.

My solution was to swallow any errors to prevent that propagating up to Google's automatic check.