Error handling in SpringBoot kafka in Batch mode

Question

I am trying to figure out is there any way to send failed records in Dead Letter topic in Spring Boot Kafka in Batch mode. I don't want to make the records being sent in duplicate as it's consuming in batch and few are already processed. I saw this link ofspring-kafka consumer batch error handling with spring boot version 2.3.7

I thought about a use case to stop container and start again without using DLT but again the issue of duplication will come in Batch mode.

@Garry Russel can you please provide a small code for batch error handling.

Answer 1

The RetryingBatchErrorHandler was added in spring-kafka version 2.5 (which comes with Boot 2.3).

The listener must throw an exception to indicate which record in the batch failed (either the complete record, or the index in the list).

Offsets for the records before the failed one are committed and the failed record can be retried and/or sent to the dead letter topic.

See https://docs.spring.io/spring-kafka/docs/current/reference/html/#recovering-batch-eh

There is a small example there.

The RetryingBatchErrorHandler was added in 2.3.7, but it sends the entire batch to the dead letter topic, which is typically not what you want (hence we added the RetryingBatchErrorHandler).