Search code examples
javamemory-leaksapache-kafkakafka-consumer-apijava-threads

KafkaConsumer.close() Why?


I am here asking about closing a kafka consumer. Do I need to close kafka consumer even if the using thread exits? Would not closing it leak resources by any change?

Here is a code example:

public class MyThread extends Thread{

    private KafkaConsumer<String, Message> kafkaConsumer;


    @Override
    public void run() {
        kafkaConsumer = initConsumer();
        while(true){
            kafkaconsumer.poll(1000000)
            //Code goes here.
        }
    }
}

Does the kafkaConsumer close when MyThread exits using System.exit?


Solution

  • Quoting from "Kafka - the definitive guide" By Gwen Shapira, Neha Narkhede, Todd Palino (O' Reilly Media) :

    Always close() the consumer before exiting. This will close the network connections and sockets. It will also trigger a rebalance immediately rather than wait for the group coordinator to discover that the consumer stopped sending heartbeats and is likely dead, which will take longer and therefore result in a longer period of time in which consumers can't consume messages from a subset of the partitions.

    Kafka - The Definitive Guide