Message Routing in AMQP

Question

I'd like to do some routing magic with AMQP. My setup is Python with Pika on the consumer/producer side and RabbitMQ for the AMQP server.

What I'd like to achieve:

1. send a message to a single exchange
2. (insert magic here)

3. consume messages like so:

   • one set of subscribers should be able to retrieve based on a routing key

   • one set of subscribers should just get all messages.

     The tricky part is that if the any server in the second set has received a message no other server from the second set will receive it. All the servers from the first set should still be able to consume this message.

Is this possible with a single basic_publish call or do I need to send the message to a routing exchange (for the first set of consumers) and to a "global" exchange for the second set of consumers?

CLARIFICATION:

What I'd like to achieve is a single call to publish a message and have it received by 2 distinct sets of consumers.

Case 1: Just receive messages based on routing key (that is a message with routing key foo will be received by all the consumers currently interested in that topic)

Case 2: This basically resembles the RabbitMQ Tutorial for Worker Queues. There are a number of workers that will receive messages dispatched in a round robin way. Only one worker will receive a message

Still the message that is received by the consumers that are interested in a certain routing key should be exactly the same as the messages received by the workers, produced by a single API call.

(Hope my question makes sense I'm not too familiar with AMQP terms)

Answer 1

To start with, you need to use a topic exchange and publish your messages with a different routing key for each queue. The magic happens when the consumer binds the queue with a binding key (or pattern to be matched). Some consumers just use the routing keys as their binding key. But the second set will use a wildcard pattern for their binding key.

For Case 1, you need to create a queue per consumer, and bind each queue with an appropriate routing key.

For Case 2, just create a single queue with a routing key of # and have each of your worker consumers consume from that. The broker will dispatch in a round-robin manner to the workers.

Here's a screenshot of what it would look like in RabbitMQ. In this example there are two consumers from your "case 1" (Foo and Bar) and one queue for all the workers to satisfy "case 2".

This model should be supported by all AMQP-compliant brokers and wouldn't require any vendor-specific enhancements.