Determine concurrency based on a key (such as an id) with RabbitMQ

Question

We are building a web application using RabbitMQ and Spring's listener-containers to produce concurrency as follows:

<rabbit:listener-container connection-factory="connectionFactory" concurrency="10">
      <rabbit:listener ref="FooService" method="handleFoo" queue-names="fooQueue"/>
</rabbit:listener-container>

<rabbit:topic-exchange name="exchange">
    <rabbit:bindings>
        <rabbit:binding queue="fooQueue" pattern="foo.handle"/>
    </rabbit:bindings>
</rabbit:topic-exchange>

I want the listeners to process messages concurrently ( e.g. 10 threads for this example ), but I don't want them to concurrently process messages with same data. For example if I am sending id's of Foo objects, I only want different Foo objects to be concurrently processed, but same Foo objects should be sequentially processed.

I have gone over the exchange and queue types of RabbitMQ but could not figure out to do this with any of them.

One way I can think of is to create multiple queues with different patterns such as foo.handle.1, foo.handle.2 and so forth. And then hash the id of the Foo objects to these patterns. But doing this for every type of queue we have and managing all of it can get out of hands quite easily.

Is there a mechanism to accomplish this with RabbitMQ?

Answer 1

Unlike JMS, RabbitMQ (or AMQP itself) has no concept of a message selector - you can't pull selective messages from a queue.

The only solution with RabbitMQ is a separate queue for each type and a single consumer on each queue.