Search code examples
architecturemessage-queuemessaging

Can someone explain what message brokers are used for?


In my line of work it's hard to go five minutes without someone extolling the virtues of MQ Series or MSMQ or the like, and I always wonder, after the sparkle of buzzwords has passed, what are some actual examples of these wonderful devices out in the real world.

What I'm looking for is something that might inspire me to find a use for one of these or give me some kind of metric I can use to evaluate a message bus/message broker/message queue -- hell, even something that will explain what the differences are between the aforementioned message* things.


Solution

  • Message queuing solutions like MQ Series or MSMQ are used extensively for integrating distributed enterprise applications, especially running on different platforms. Done right (with persistent queues, asynchronous design rather than 'RPC over MQ' and attention to operational requirements), this gives you high availability compared to synchronous request/reply integrations such as RPC or boilerplate web services (whose availability is the product of the respective availabilities: integrating ten systems with 99 % availability synchronously gives you a combined availability of no more than 90 % -- or worse if there's just one weak link). Mind you, this requires the message queues to have high availability in themselves: we use our mainframe for that purpose (take a guess at which product we're using!).

    Message (or integration) brokers and 'buses' are a more complex kettle of fish. They can add

    • translation between different content representations (text encodings and code pages)
    • supervision, detecting when a target system does not pick up queued messages and raising alarms or automatically restarting
    • transformation, when systems do not 'speak the same language' and represent e.g. customer or product records differently: this can in principle help you deploy new versions at different rates
    • routing (up to and including publish/subscribe) to decouple a sending system from knowledge of the details of recipients, thus reducing impact of changes in target systems
    • orchestration, where you can coordinate messages between a number of systems to track a longer real-life business process (say, from customer order to delivery to invoicing).

    I've listed these functionalities in roughly increasing order of difficulty (and potential reward). The higher you get, the more mature your organization (including the business side) needs to be in order to gain the advantages.