Search code examples
node.jsapache-kafkakafkajs

KafkaJS consume from multiple topics with one consumer


I want to implement retry logic while consuming from Kafka topic using KafkaJS, so basically, I will have 2 topics main-topic and retry-topic and I will

read from -> main-topic
if processing fails | -> retry topic

so is it a bad practice to use one consumer for listening from both topics(both main and retry), as kafka allows to listen from multiple topics using same consumer.


Solution

  • It's not a bad practice at all.

    The only problem you may run into using one consumer is that the topics may need differ configurations (connection settings, deserializer, etc). In that case, you can create two separate Consumer instances rather than one subscribing to both.