Search code examples
celery

celery execute all tasks in queue in certain time

I have a tasks that makes connection to database each time and it can be executed like 1000 times a day. so i don't want this task to be executed instantly when it pushed to the queue.

instead I want all tasks in this queue to wait for certain time (e.g. 01:00 AM) then start executing tasks one by one.

other than that I have some kind of routing and priorities that I still want to be done.

CELERY_TASK_ROUTES= {
'report_app.tasks.*': {'queue': 'create_report_queue', 'priority': 1},
'link_app.tasks.*': {'queue': 'add_link_queue', 'priority': 2},
}

I use Rabbitmq as broker.

Solution

  • The simplest way to do what you have to do is to use the task ETA. The ETA (estimated time of arrival) lets you set a specific date and time that is the earliest time at which your task will be executed.

    Example:

    from datetime import datetime, timedelta

tomorrow = datetime.utcnow() + timedelta(days=1)
add.apply_async((2, 2), eta=tomorrow, queue="report_queue")