Search code examples
djangocelerycelerybeat

Celery Schedule, can't pickle nowfun


I'm trying to configure Celery to run different tasks in different time zones maintaining the shift in daylight savings. I set up my task like this:

import datetime
import pytz
from celery import Celery
from celery.schedules import crontab

app = Celery('my_project')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

NOW_MT = lambda: datetime.datetime.now(pytz.timezone('America/Denver'))

app.conf.beat_schedule = {
    "My Daily Mountain Time Task": {
        "task": "app.tasks.some_task",
        "schedule": crontab(minute='0', hour='0', nowfun=NOW_MT)
    },
    "My Daily UTC Task": {
        "task": "app.tasks.some_other_task",
        "schedule": crontab(minute='0', hour='0')
    }
}

But my celery beat process is erring out with the following message:

Can't pickle <function <lambda> at 0x7f71055a4670>: attribute lookup <lambda> on app.celery failed

Solution

  • Use the function instead of lambda functions

    import datetime
    import pytz
    from celery import Celery
    from celery.schedules import crontab
    
    app = Celery('my_project')
    app.config_from_object('django.conf:settings', namespace='CELERY')
    app.autodiscover_tasks()
    
    def NOW_MT():
        return datetime.datetime.now(pytz.timezone('America/Denver'))
    
    app.conf.beat_schedule = {
        "My Daily Mountain Time Task": {
            "task": "app.tasks.some_task",
            "schedule": crontab(minute='0', hour='0', nowfun=NOW_MT)
        },
        "My Daily UTC Task": {
            "task": "app.tasks.some_other_task",
            "schedule": crontab(minute='0', hour='0')
        }
    }