Search code examples
apache-kafkaapache-kafka-streams

Does Kafka-streams commit offset if an exception was thrown during processing?

I have a Kafka streams application, and I created custom production/consumption/uncaught exception handlers, all return CONTINUE to continue processing despite the exception. Does Kafka stream commit the offset in this case? I looked at the code, and it seems like it doesn't, but this does not make much sense to me.

Solution

  • Internally, the Streams API leverages Kafka’s consumer client to read input topics and to commit offsets of processed messages in regular intervals using the value set in commit.interval.ms.

    In case of the exception handler, each exception handler can return a FAIL or CONTINUE depending on the record and the exception thrown. Returning FAIL will signal that Streams should shut down and CONTINUE will signal that Streams should ignore the issue and continue processing.

    Either case, Commit on the offset will be performed internally by the Streams API.