I have a very simple use case. I have one Kafka Consumer and I want to move all invalid or partial messages to a dead letter queue. The example from the documentation uses SeekToCurrentErrorHandler and to attach DeadLetterPublishingRecoverer. In my case, I don't want to retry these invalid messages and I have set the maxFailures to 1 (I have tried 0 as well with the same result). The problem here is that for some reason SeekToCurrentErrorHandler seeks the partition every time I have an invalid message even that I just want to move it to DLT and this makes the whole process very slow. I'm not sure this is the right behavior from SeekToCurrentErrorHandler, but still is there a better way to achieve my goal without SeekToCurrentErrorHandler? Do I have to create custom ErrorHandler?
P.S.
The strange behavior of SeekToCurrentErrorHandler is observed when there are more invalid messages then the consumer can buffer. If there are a few messages everything is fast, but when we have a big chunk of invalid messages it performs awful.
This was a bug - it is fixed on master and 2.2.x. It will be in the 2.2.5 release due next week.
The
SeekToCurrentErrorHandlerandDefaultAfterRollbackProcessoralways retried at least one time, even ifmaxAttemptswas 1.