GAE Getting lots of CancellationException for Task Queue DataStore call

Question

I have a Java GAE instance that runs TaskQueue jobs. In the last 48 hours I have been getting lots of CancellationException errors (about 36 in a 24 hour period).

The error always occurs on a DataStore query to a particular table. The query is on an indexed column, projectId,and the query is a composite OR (i.e. projectId = -1 or projectId = 23). The entire table in question has 96 records and any one query will pull back 20 records at most.

Does anyone have any idea why I'm getting these problems

Thanks Paul

Update: Sorry my mistake, this isn't part of a Task Queue call it is the code that sets up the task queue call. The time the call took is on average about 59.5 secs (thanks for pointing that out Alex). Looking at the trace (thanks Alex) it seems that the delay is in the datastore call. It can be as quick as 21ms or over 59000ms, which is way to long. I just checked the logs again and there hasn't been any of those exceptions in the last 24 hours so this might be a 48 hour glitch.

Answer 1

Understanding "CancellationException: Task was cancelled" error while doing a Google Datastore query

Based on this question, it sounds like CancellationException means that you are hitting the 10-minute timeout that TaskQueue Jobs have to complete execution.

Do your logs show how long these requests are running for before they throw an exception (for example stackdriver shows that this deferred task ran for 902ms)?

Querying a table of 96 records should be fine, but it's hard to be certain without being able to see what the code of your task looks like.

You could split your task up into smaller 'sub-tasks' and have sub-task launch the next sub-task before returning.

EDIT: you could also try going here https://console.cloud.google.com/traces/traces to see a break down of where your request is spending its time