Search code examples
phpduplicatesrabbitmqamqp

How to prevent duplicates in RabbitMQ dead lettering?


Firstly, sorry for the bad title! I couldn't find anything specific that matched this issue, and can't find anything on Google or Stackoverflow regarding this problem.

So first some background, I have a RabbitMQ consumer based on php-amqplib\Thumper and succesful message consumption is all working fine and as expected.

However, I have a method which is the callback for RabbitMQ everytime it dispatches a message to the consumer, this callback is fired.

I have it wrapped around a try catch, so that, if a message fails to process, I can set a retry_count header, once that message comes back round a third time, e.g. retry_count: 3 I publish it to a dead queue, and send a basic_nack.

However, here is where my problem lies, I'm getting more than 1 message per message in my dead queue, i.e. for every failed message, 2 appear in my dead queue, one which has the correct dead queue properties. e.g.

delivery_mode: 2 headers:
x-death:
count: 1 reason: rejected queue: ee.api.events.medusa time: 1498730321 exchange: EES_ApiEvents routing-keys: ee.api.events.medusa.WALLET.UPDATE

The other, just has delivery_mode and no other properties, and I cannot figure out for the life of me where this superfluous message is coming from!

This is the bit of code that is run when the retry_count reaches 3:

        $producer
            ->setDurableExchange(true)
            ->setExchangeType(Producer::RABBITMQ_EXCHANGE_TYPE_TOPIC)
            ->setExchangeAutodelete(false)
            ->setExchangeName($deadExchangeName)
            ->setQueueName($deadQueueName)
            ->setRoutingKey($routingKey)
            ->setBindings([
                $routingKey,
            ]);

        $producer->publish(Json::decode($message->body, true));

        $message->delivery_info['channel']->basic_nack($deliveryTag, false, false);

Thanks in advance!


Solution

  • So after tons of playing around, I went back to the documentation and reread, and it clearly states, if you reject a message with requeue=false then it will automatically be dead lettered if, you defined a dead letter exchange against the main queue.

    So problem is now solved!