So I have a task that creates a work directory and does all its work there. The task is called from server A and executed on worker servers.
I need to make sure the work directory is deleted once the task is done/canceled.
I added a task revoked handler and it looks like this:
@task
def my_task(value):
task_id = current_task.request.id
work_dir = os.path.join(BASE_WORK_DIR, task_id)
os.makedirs(work_dir)
try:
# Do work...
finally:
shutil.rmtree(work_dir)
@task_revoked.connect(sender=my_task)
def my_task_revoked_handler(*args, **kwargs):
# FIXME: delete work_dir
print args
# ()
print kwargs
# {'terminated': True, 'signal': <Signal: Signal>, 'expired': False, 'sender': <@task: myapp.core.tasks.my_task>, 'signum': '15'}
My problem is when server A cancels the task I cannot issue a cleanup of the work directory in the revoked handler since it doesn't have the task_id.
Is there any way to get the task id from this specific signal handler? Some other Signals have them, and I've looked at the source to where these are issued and for some reason this Signal isn't provided with the task_id.
The provided sender task contains a trace_task function: {'__trace__': <function trace_task at 0x3ee8230>} but I can't see how I could use it since the function itself requires a task_id.
Any other ideas are welcome.
I think what's happening here is you're using an older version of celery that doesn't have support for this first "request" argument.
The upstream issue that added this was [1]; prior to that, I think that you're out of luck and there is no way to get the task_id unfortunately.