Search code examples
pythonceleryamqp

Celery: restrict queues for a task

I have multiple queues, let's say these are Q1 and Q2.

I also have some tasks, one of them is T1.

How do I configure Celery so that a worker only takes T1 from Q1, and never accepts T1 from Q2?

Solution

  • You can achieve this with celery apps. Register different types of tasks to different apps and run workers on those apps only (eg. see -A myapp setting on celery worker command).

    Alternatively you might also have success if you go down the simpler route of using different queues. So if you manage to send T1 only to queue named queue1 and T2 to queue2, you can use option -Q queue1 on celery worker so it only takes tasks from queue1 for example.