How can I make only one instance of an application receive each AMQP topic message (consumer group behaviour)

I am using Apache Camel's AMQP component to listen to messages from an ActiveMQ Artemis topic.

This application is run on Kubernetes with two replicas.

I've configured a durable subscription with a unique clientId per pod and a common subscription name:

<route autoStartup=true" id="myRoute">
    <from id="_amqp_topic" uri="amqp:topic:xxx?connectionFactory=#amqpCF&amp;disableReplyTo=true&amp;transacted=false&amp;subscriptionDurable=true&amp;clientId={{container-id}}&amp;durableSubscriptionName=eventSubscription"/>
    <log loggingLevel="INFO" message="Received event: ${body}"/>
    ...
</route>

The problem is both pods receive the message, when only one of them should. I'm trying to achieve something similar to Kafka's consumer groups, where only one member of the group receives each message.

Solution

If you want only one subscriber to receive the message then the subscribers must share the same subscription. Therefore, you need to:

set subscriptionShared=true in your _amqp_topic uri
use the same client ID (i.e. don't use {{container-id}})

For example:

<route autoStartup=true" id="myRoute">
    <from id="_amqp_topic" uri="amqp:topic:xxx?connectionFactory=#amqpCF&amp;disableReplyTo=true&amp;transacted=false&amp;subscriptionDurable=true&amp;clientId=myClientID&amp;durableSubscriptionName=eventSubscription&amp;subscriptionShared=true"/>
    <log loggingLevel="INFO" message="Received event: ${body}"/>
    ...
</route>

Another option would simply to use a queue instead of a topic, e.g.:

<route autoStartup=true" id="myRoute">
    <from id="_amqp_queue" uri="amqp:queue:xxx?connectionFactory=#amqpCF&amp;disableReplyTo=true&amp;transacted=false"/>
    <log loggingLevel="INFO" message="Received event: ${body}"/>
    ...
</route>