Search code examples
pythonceleryschedulercelerybeatdjcelery

python celery - how to add CELERYBEAT_SCHEDULE task at runtime to a worker?


I have created a celery worker with a single celerybeat schedule task which runs at 5 seconds time interval. How can I add another beat task dynamically to the celery worker without stopping it?

Example

app.conf.update(
   CELERY_TASK_RESULT_EXPIRES=3600,
   CELERY_TIMEZONE = 'UTC',
   CELERYBEAT_SCHEDULE = {
    'long-run-5-secs': {
        'task': 'test_proj.tasks.test',
        'schedule': timedelta(seconds=5),
        'args': (16, )
    }
   }
)

With the above configuration, I am able to run the celery worker with beat mode successfully.

Now I need add the below beat schedule dynamically:

'long-run-2-secs': {
    'task': 'test_proj.tasks.test',
    'schedule': timedelta(seconds=2),
    'args': (14, ) },

Thanks


Solution

  • I've been looking for solution for the very same problem. I am affraid you'll have to wait for Celery ver.4.0. Dynamic task scheduling is only currently supported in the development version : http://docs.celeryproject.org/en/master/userguide/periodic-tasks.html#beat-entries