Search code examples
apache-kafkaspring-kafka

Can two synchronous clients use same request/reply topic


I have a User service which is listening to a request topic and returning the User object. I need to call this service synchronously from two different services and wanted to confirm if its ok for them to use the same request/response topic names to both request the User object?


Solution

  • See the documentation.

    There are two options when using the same reply topic:

    1. Discard unexpected replies:

    When configuring with a single reply topic, each instance must use a different group.id. In this case, all instances receive each reply, but only the instance that sent the request finds the correlation ID. This may be useful for auto-scaling, but with the overhead of additional network traffic and the small cost of discarding each unwanted reply. When you use this setting, we recommend that you set the template’s sharedReplyTopic to true, which reduces the logging level of unexpected replies to DEBUG instead of the default ERROR.

    1. Use dedicated partitions:

    If you have multiple client instances and you do not configure them as discussed in the preceding paragraph, each instance needs a dedicated reply topic. An alternative is to set the KafkaHeaders.REPLY_PARTITION and use a dedicated partition for each instance. The Header contains a four-byte int (big-endian). The server must use this header to route the reply to the correct partition (@KafkaListener does this). In this case, though, the reply container must not use Kafka’s group management feature and must be configured to listen on a fixed partition (by using a TopicPartitionOffset in its ContainerProperties constructor).