Celery beat not sending crontab task when hour is set

Question

I'm using celery 4.1 and all my periodic tasks work correctly except where I set the hour in a crontab task. I was thinking it had something to do with the timezone setting, but I can't seem to work out where the problem is.

dashboard/celery.py

from __future__ import absolute_import, unicode_literals
from celery import Celery

app = Celery('dashboard',
         broker='redis://',
         backend='redis://localhost',
         include=['dashboard.tasks'])

app.conf.update(
    result_expires=3600,
    enable_utc = False,
    timezone = 'America/New_York'
)


if __name__ == '__main__':
    app.start()

This works:

@app.task
@periodic_task(run_every=(crontab()))
def shutdown_vms():
    inst = C2CManage(['stop','kube'])
    inst.run()
    return

This works:

@app.task
@periodic_task(run_every=(crontab(minute=30,hour='*')))
def shutdown_vms():
    inst = C2CManage(['stop','kube'])
    inst.run()
    return

This doesn't work:

@app.task
@periodic_task(run_every=(crontab(minute=30,hour=6)))
def shutdown_vms():
    inst = C2CManage(['stop','kube'])
    inst.run()
    return

Beat picks up the task just fine:

<ScheduleEntry: dashboard.tasks.shutdown_vms dashboard.tasks.shutdown_vms() <crontab: 30 6 * * * (m/h/d/dM/MY)>

But it never sends it. I've let the processes run over a weekend and it never submits the task. I don't know what I'm doing wrong. I do have other tasks that run on timedelta periodicity and they all work perfectly.

Any help would be awesome.

EDIT: host is set to use the America/New_York timezone.

EDIT2: running beat as a separate process:

celery -A dashboard worker -l info

celery -A dashboard beat -l debug

I run them detached mostly or use multi.

Answer 1

Looks like this bug is causing it.

https://github.com/celery/celery/issues/4177

And several others that indicate that scheduling is not calculated properly when not using UTC.

Switched celery to use UTC as timezone and enabled utc and it works fine.