Search code examples
rabbitmqmessage-queuepriority-queue

RabbitMQ 2 Queues


Saw similar questions but different expected answers. My question is can I can create a consumer to focus on a single queue until it empties then switch to the other queue, until new work is sent to their main queue?

For example: 1 queue contains large amount of work to be processed in longer time frame and its own dedicated consumers (3 for instance). The 2nd queue receives much less work that requires less processing. If the consumers for the 2nd queue complete their work can I make it so they switch to the first queue until their queue receives more work?


Solution

  • I think for this question, it's important to keep in mind that there is a difference between a "consumer" in the canonical sense vs. a "consumer" in the RabbitMQ sense.

    A RabbitMQ Consumer is a contrivance of the protocol - basically, it is a designation that the channel/connection would like to have messages pushed to it, under a designated consumer tag. In this sense, it is merely a notification to the broker to immediately send messages.

    In the canonical sense, a message consumer is any piece of code that processes messages.

    So, the answer to your question is "yes, go ahead and write your program to do that." You have control over the canonical consumer code. It is up to your software to determine what to do with a message that arrives from a queue.

    Now, if you're wondering if RabbitMQ can re-subscribe a consumer to a different queue, the answer is "that's not how it works." In RabbitMQ, a consumer is simply a response to a request to subscribe to a queue - it is a "consumer tag" object. The ongoing nature of the subscription is tied to the channel/connection pair.

    What should you do? While your code doesn't specify any particular coding language, in my opinion, you're off-track by even asking this question. Subscribe to both queues. If there is nothing for the worker to do, I think the computer would be perfectly happy with that. If you're worried about a particularly busy queue issuing too much work, you can use a number of techniques to throttle messages coming into that consumer. One popular technique is prefetch.