Search code examples
rabbitmqrabbitmq-shovel

How to configure RabbitMQ bidirectional shovel and avoid infinite loop?


I have 2 RabbitMQ clusters in separate datacenters. At the moment we are shovelling one of the exchanges from an one rabbit to the other but now we need to shovel in the other direction too. How can we achieve this without causing an infinite loop?

A colleague thought we could tick the "Add Forwarding Headers" options and then filter using a Header exchange. However, the headers that get added appear to be nested and I've not found a way to match against the headers?


Solution

  • Seems based on the additional description you provided that:

    • You are using shovel between exchanges
    • Guessing that the exchange type is fanout

    I can see two approaches to handle what you're trying to do:

    • Use federation, and configure it in both directions between the exchanges in each cluster
    • Still use shovel, but define a new queue on each exchange and configure a queue to queue shovel

    For the shovel option: You have exchange A on cluster 1, exchange A on cluster 2, your main_queue on each cluster

    • Create a secondary_queue on each cluster, bound to exchange A.
    • Create a shovel from secondary queue on cluster 1 to main_queue on cluster 2
    • Create a shovel from secondary queue on cluster 2 to main_queue on cluster 1

    Benefit of Federation: simpler, even when you have complex routing configured

    Benefit of shovel: better if you have risk of major connection disruption (more than a minute I'd say). In such cases you won't lose messages while it might happen in the Federation case that some messages don't get replicated on both clusters.