Below is the code from documentation:
from celery.exceptions import SoftTimeLimitExceeded
@celery.task(soft_time_limit=15, time_limit=20)
def mytask():
try:
return do_work()
except SoftTimeLimitExceeded:
cleanup_in_a_hurry()
The question is how celery allows to catch the exception inside the function.
If it executes my_task and raises SoftTimeLimitExceeded, how is this exception propagates inside the function?
Also, why it is not possible to catch TimeLimitExceeded inside the function?
Thank you.
In short, Celery interrupts your task via a signal and raises SoftTimeLimitExceeded.
Note that there are a few different ways Celery can be configured to run tasks (e.g. threads), but my answer is limited to process pools. In this case, your task is being executed in one of the worker's child processes. When these pool processes are created, Celery registers a signal handler to handle the SIGUSR1 signal. Upon timeout, SIGUSR1 is sent. This interrupts your tasks: wherever they were in their Python bytecode execution, they stop, add Celery's signal handler onto their stack frames, and execute Celery's handler, which raises SoftTimeLimitExceeded. The exception propagates up the stack frame (note this will be wherever your task was when the interruption occurred) until it is caught, presumably by your task code.