I am trying to understand the logic for message deletion in RabbitMQ.
My goal is to make messages persist even if there is not a client connected to read them, so that when clients reconnect the messages are waiting for them. I can use durable, lazy queues so that messages are persisted to disk, and I can use HA replication to ensure that multiple nodes get a copy of all queued messages.
I want to have messages go to two or more queues, using topic or header routing, and have one or more clients reading each queue.
I have two queues, A and B, fed by a header exchange. Queue A gets all messages. Queue B gets only messages with the "archive" header. Queue A has 3 consumers reading. Queue B has 1 consumer. If the consumer of B dies, but the consumers of A continue acknowledging messages, will RabbitMQ delete the messages or continue to store them? Queue B will not have anyone consuming it until B is restarted, and I want the messages to remain available for later consumption.
I have read a bunch of documentation so far, but still have not found a clear answer to this.
The simple answer is that messages consumed from one queue have no bearing on messages in another. Once you publish a message, the broker distributes copies to as many queues as appropriate - but they are true copies of the message and are absolutely unrelated from that point forward so far as the broker is concerned.
Messages enqueued into a durable queue remain until they are pulled by a consumer on the queue, and optionally acknowledged.
Note that there are specific queue-level and message-level TTL settings that could affect this. For example, if the queue has a TTL, and the consumer does not reconnect before it expires, the queue will evaporate along with all its messages. Similarly, if a message has been enqueued with a specific TTL (which can also be set as a default for all messages on a particular queue), then once that TTL passes, the message will not be delivered to the consumer.
Secondary Note In the case where a message expires on the queue due to TTL, it will actually remain on the queue until it is next up to be delivered.