Set polling rate for kafka consumers per topic using springboot

Question

Is it possible to set different poll rates for consumers each kafka topics on SpringBoot? I would like to have one topic to be polling with longer intervals (like 5 minutes). I was hoping to use this to implement a retry for failed processing of messages from kafka.

A sample implementation will greatly help.

Answer 1

Since version 2.3, the listener container has a new property idleBetweenPolls. The container will ensure that the time between polls is no larger than max.poll.interval.ms - 5000.

You can set it for specific consumers using a container customizer bean...

@Component
class Customizer {

    public Customizer(ConcurrentKafkaListenerContainerFactory<?, ?> factory, DefaultKafkaProducerFactory<?, ?> pf) {
        factory.setContainerCustomizer(container -> {
            if (container.getContainerProperties().getGroupId().equals("slowGroup")) {
                // or you can use the topic(s)
                container.getContainerProperties().setIdleBetweenPolls(60_000);
            }
        });
    }

}