I've followed the celery doc https://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html to create an app and periodic tasks as below:
$ tree demo/
demo/
├── config.py
├── __init__.py
└── tasks.py
$ cat demo/__init__.py
# -*- coding: utf-8 -*-
from celery import Celery
app = Celery('demo')
app.config_from_object('demo.config')
$ cat demo/config.py
# -*- coding: utf-8 -*-
BROKER_URL = "redis://127.0.0.1:6379"
CELERY_TIMEZONE='UTC'
CELERY_IMPORTS = [
"demo.tasks",
]
$ cat demo/tasks.py
from demo import app
@app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs):
sender.add_periodic_task(3.0, say.s(), name='say hello every 3s')
@app.task
def say():
print("Hello!")
And then run celery beat as below:
$ celery beat -A demo -l info --max-interval 10
celery beat v4.3.0 (rhubarb) is starting.
__ - ... __ - _
LocalTime -> 2019-12-12 16:26:41
Configuration ->
. broker -> redis://127.0.0.1:6379//
. loader -> celery.loaders.app.AppLoader
. scheduler -> celery.beat.PersistentScheduler
. db -> celerybeat-schedule
. logfile -> [stderr]@%INFO
. maxinterval -> 10.00 seconds (10.0s)
[2019-12-12 16:26:41,234: INFO/MainProcess] beat: Starting...
Wait for a while, there isn't any task scheduled.
However if I change to set CELERYBEAT_SCHEDULE
in config, it can work well. I do that by changing demo/config.py
and demo/tasks.py
as below:
$ cat demo/config.py
# -*- coding: utf-8 -*-
from datetime import timedelta
BROKER_URL = "redis://127.0.0.1:6379"
CELERY_TIMEZONE='UTC'
CELERY_IMPORTS = [
"demo.tasks",
]
CELERYBEAT_SCHEDULE = {
'say hello every 10 seconds': {
'task': 'demo.tasks.say',
'schedule': timedelta(seconds=3),
},
}
$ cat demo/tasks.py
from demo import app
@app.task
def say():
print("Hello!")
Then run celery beat with the same command as before, the periodic tasks can be scheduled every 3 seconds as expected.
What's wrong with my previous setup?
FYI, I figured out another solution, changing the decorator @app.on_after_configure.connect
to @app.on_after_finalize.connect
can make it work. Though I don't know the exact reason at this moment.