Search code examples
apache-kafkaapache-kafka-streams

Does a Kafka Streams app commit offets for a topic used to fill a Global KTable?


I'm observing my Kafka Streams app reporting consumer lag for topics used to fill Global KTables. Is it correct that offsets are not committed for such topics?

This would make sense as the topic is read from the beginning on every startup, so keeping track of the offest in the consumer would be sufficient.

It would however be useful to know for monitoring to exclude such consumer topic pairs.


Solution

  • Correct, offsets are not committed for "global topics" -- the main reason is, that all KafkaStreams instances read all partitions and committing multiple offsets is not possible.

    You can still access the "global consumer" metrics, that also report their local lag.