According to the internet (my question is at the bottom), when using the drop-head overflow behavior in RabbitMQ, it's important to understand that this behavior is distinct from the mechanisms that route messages to a dead-letter exchange (DLX). The drop-head behavior is specifically related to how a queue handles situations when it reaches its maximum length.
Here's a breakdown of what happens:
Drop-Head Behavior: When a queue is configured with the drop-head overflow behavior and it reaches its maximum length, the oldest messages in the queue (those at the "head" of the queue) are dropped to make room for new incoming messages. This means that the oldest messages are simply removed from the queue.
Dead-Letter Exchange Routing: Normally, messages are routed to a DLX under specific conditions, such as message rejection (nack), message expiration, or the queue reaching a maximum length. However, this routing to a DLX is a separate mechanism. Not all actions that remove messages from a queue send them to a DLX. In the case of drop-head, the messages are not dead-lettered but are simply discarded.
No Dead-Lettering for Dropped Messages: When messages are dropped due to the drop-head overflow behavior, they are not sent to a DLX. They are effectively lost. This is a crucial point to consider when designing your message handling architecture, especially in systems where losing messages can have significant consequences.
In summary, the drop-head overflow behavior in RabbitMQ leads to the oldest messages being dropped and not sent to a DLX. This is in contrast to other conditions (like message rejection or expiration) where messages can be dead-lettered and rerouted. It's important to configure your RabbitMQ setup and overflow behavior according to your application's specific needs and tolerance for message loss.
My questions:
Is there any way to configure RMQ to not just let messages slip to the void when "drop-head" or "drop-tail" is used and instead send them somewhere else?
Likewise, I would be curious about how to automatically put messages on a secondary queue/exchange, if the "mandatory" publishing option is set to true and no queues match the routing key
I don't think there are any options doing exactly what you described, but you can try using reject-publish-dlx. While using this, the most recent messages will be discarded if the queue overflows.
From the docs:
If overflow is set to reject-publish or reject-publish-dlx, the most recently published messages will be discarded. ... If a message is routed to multiple queues and rejected by at least one of them, the channel will inform the publisher via basic.nack. The message will still be published to all other queues which can enqueue it. The difference between reject-publish and reject-publish-dlx is that reject-publish-dlx also dead-letters rejected messages.
I think the only option here is an alternate exchange. If AE is configured on the queue and the message is published to the queue without any matching routing keys, it will be routed to your alternate exchange.
From the docs:
Whenever an exchange with a configured AE cannot route a message to any queue, it publishes the message to the specified AE instead. If that AE does not exist then a warning is logged. If an AE cannot route a message, it in turn publishes the message to its AE, if it has one configured. This process continues until either the message is successfully routed, the end of the chain of AEs is reached, or an AE is encountered which has already attempted to route the message.
There are examples in the docs for configuring an AE through policies, or through client-provided arguments.