Search code examples
architecturemessaging

Local vs. remote queues in pub/sub messaging

If I was building a system with dozens of publishers/subscribers using message queues, it seems I have a few network configuration options:

  1. I could have one clustered broker that all machines use - each machine would not have a local queue
  2. I could install brokers locally on every machine, and use store-and-forward to deliver messages to remote machines

Different technologies seem to enforce different configurations - e.g., MSMQ requires every machine to have its own local queue, while Tibco EMS seems to often be used in clusters, without local queues on each consumer.

What are the downsides to not having a local queue, and what factors influence the decision?

Solution

  • Not having a local queue that provides a durable message store means that you cannot guarantee message delivery. Using something like RabbitMQ in a cluster with a broker instance locally gives you a durable mechanism to store messages for delivery. If you have to connect over a network connection to a remote broker to send a durable message, you're at a higher risk of network failure.

    MSMQ is also store-and-forward, but it doesn't provide any clustered routing capabilities. This means that the application has to do the work (or have a layer on top of it, such as MassTransit or NServiceBus do it for you).

    When I think of TIBCO, I think of a centralized cluster of EMS servers that application servers communicate with instead of running a broker instance locally. The GUI tools that wrap around EMS and the BusinessWorks application servers really force a model in that world.

    In any of the distributed cases where messages are stored locally, it's important to make sure the machine itself is properly equipped for message storage, with fast disk and sufficient disk for the expected message backlog/capacity.