How to dynamically add / remove periodic tasks to Celery (celerybeat)

Question

If I have a function defined as follows:

def add(x,y):
  return x+y

Is there a way to dynamically add this function as a celery PeriodicTask and kick it off at runtime? I'd like to be able to do something like (pseudocode):

some_unique_task_id = celery.beat.schedule_task(add, run_every=crontab(minute="*/30"))
celery.beat.start(some_unique_task_id)

I would also want to stop or remove that task dynamically with something like (pseudocode):

celery.beat.remove_task(some_unique_task_id)

or

celery.beat.stop(some_unique_task_id)

FYI I am not using djcelery, which lets you manage periodic tasks via the django admin.

Answer 1

No, I'm sorry, this is not possible with the regular celerybeat.

But it's easily extensible to do what you want, e.g. the django-celery scheduler is just a subclass reading and writing the schedule to the database (with some optimizations on top).

Also you can use the django-celery scheduler even for non-Django projects.

Something like this:

• Install django + django-celery:

  $ pip install -U django django-celery

• Add the following settings to your celeryconfig:

  DATABASES = {
      'default': {
          'NAME': 'celerybeat.db',
          'ENGINE': 'django.db.backends.sqlite3',
      },
  }
  INSTALLED_APPS = ('djcelery', )
  

• Create the database tables:

  $ PYTHONPATH=. django-admin.py syncdb --settings=celeryconfig
  

• Start celerybeat with the database scheduler:

  $ PYTHONPATH=. django-admin.py celerybeat --settings=celeryconfig \
      -S djcelery.schedulers.DatabaseScheduler
  

Also there's the djcelerymon command which can be used for non-Django projects to start celerycam and a Django Admin webserver in the same process, you can use that to also edit your periodic tasks in a nice web interface:

   $ djcelerymon

(Note for some reason djcelerymon can't be stopped using Ctrl+C, you have to use Ctrl+Z + kill %1)