Search code examples
pythondjangorabbitmqcelerycelerybeat

A finite celery beat task

I'm using Celery on a project that integrates with a 3rd party API. I need to send a file to the service for processing. After I send the file I need to request the status of the processing every 10 seconds until it's complete.

As I have it now, the main task will spin off subtasks to check the status of the processing. I'd like these subtasks to act like finite celery beat tasks. I want to start the tasks life when I call it as a subtask and I want to end it when the task is complete.

What's the best way of accomplishing this?

Solution

  • Use Task.retry, for example:

    from celery.result import AsyncResult

@task
def verify_task(task_id):
    result = AsyncResult(task_id)
    if result.ready():
         do_something_with(result.get())
    else:
        verify_task.retry(countdown=1)

    The countdown is the number of seconds before the retry.