I like to use a Spring Kafka record message listener with the out of order commit feature. As processing of records may take some time due to e.g. related network calls, my idea is to submit the consumed messages into a thread pool. There the messages will be processed asynchronously and commited by calling the acknowledge method.
However I wondered how Spring Kafka behaves in the case errors while using out of order commits. The documentation states, that the nack method must not be used together with this feature. Moreover, the listener container defers the commit until all acknowledgements are received and the consumer is paused in the meanwhile.
The documentation doesn't give any hint what happens if a record is never acknowledged, e.g. because of an exception inside a thread of the pool which is passed back to the message listener.
So my question is how Spring Kafka behaves in those situations? Are the messages going to replayed or do i need to take care of the retries on my own?
Thanks for your help!
The container will be paused indefinitely; you are responsible for handling any errors in your code. The container cannot continue until all the gaps are filled.
If you stop the container with missing commits, any deferred commits will be lost and all uncommitted records will be redelivered when the container is restarted.
It is generally better to avoid out of order commits by reducing the max.poll.records to a number that can be handled in the max.poll.interval.ms.