How to implement a competing consumer patter with rebus on msmq

Question

Is there any way to implement a competing consumer patter on MSMQ with Rebus ?

I was hoping to just have consumer point at a single remote queue, but that doesn't seem to work.

Thanks

Matt

Answer 1

Since MSMQ queues are local on a machine, and since transactional remote receive is pretty inefficient, there's no point in doing real competing consumers with MSMQ.

This is also why configuring a remote queue as an endpoint's input queue is simply not possible with Rebus.

What you can do, is to use Rebus' transport message routing capabilities and insert a router, which will function as a round-robin load balancer, to forward messages to the individual workers (which are then free to run on other machines).

Your starting point could be something like this (where queue is simply the current name of the input queue):

Without changing anything in Producer, you can install a Router to receive messages from queue, using the aforementioned transport message forwarding capabilities to forward each message to one of the consumers:

The Router could then be implemented with something like the following piece of code:

var consumerQueueNames = new [] {
    "worker1",
    "worker2",
    "worker3"
};

var currentConsumerIndex = 0;

Configure.With(...)
    .Transport(t => t.UseMsmq("queue"))
    .Routing(r => {
        r.AddTransportMessageForwarder(async transportMessage => {
            var index = Interlocked.Increment(ref currentConsumerIndex) 
                        % consumerQueueNames.Length;

            var destination = consumerQueueNames[index];

            return ForwardAction.ForwardTo(destination);
        });
    })
    .Start();

I hope this gave you enough inspiration to implement load balancing with Rebus and MSMQ :)