Search code examples
spring-kafka

How does Spring Kafka consider a record or batch as processed?


Found this from Spring Kafka documentation:

BATCH: Commit the offset when all the records returned by the poll() have been processed.

RECORD: Commit the offset when the listener returns after processing the record.

I wanted to know how does it get considered it as "processed"? Is it the kafka listener received a message and finished without any errors? Suppose it gets stopped in between due to some reason(app stopped, some exception occurred) will it be considered processed?

Also in case of ackMode = BATCH for a non batch listener what will happen? Will it act like ackMode = RECORD? Committing one record at a time?

I wanted to confirm these behaviours are right or not.


Solution

  • "Processed" in this context means the listener exited normally; if the listener throws an exception, the behavior depends on the configured error handler. By default, the error handler will re-seek the partitions and redeliver the failed record up to 10 times (with no delay between attempts), you can modify that by configuring an appropriate back off.

    With AckMode.BATCH and a record listener, the previously processed records will be committed before calling the error handler.

    See the documentation https://docs.spring.io/spring-kafka/docs/current/reference/html/#annotation-error-handling