Search code examples
spring-kafka

Shutdown kafka consumer after processing messages

I am using @KafkaListener(topics = "${topic}") to consume messages from a topic in a spring-boot application and I need this running periodically. spring-kafka version is 2.2.4.RELEASE.

One way to achieve this could have been batching every 6 hours using fetch.max.wait.ms, but 6 hours seems too much for this configuration.

Hence, I am looking for a way to shut down the application after processing, and restart it every 6 hours.

Other way is something like below, but it does not guarantee that application had finished processing within the sleep time(30 sec in below example).

public class Application {
    public static void main(String[] args) throws InterruptedException {
        ConfigurableApplicationContext run = SpringApplication.run(Application.class, args);
        Thread.sleep(30000);
        run.close();
    }
}

What is the graceful way to shutdown the consumer, to make sure that shutdown happens only after it has processed the batch of messages?

Solution

  • See this answer.

    Shut down the application when all of the container instances go idle.