Search code examples
apache-kafkaapache-camel

Unable to Consume old messages in the kafka


The Producer in Kafka is producing messages, but the consumer is down after consuming the 5th message. After some time the consumer is up, and I want to consume the 6th message, instead of the latest message.

How can I do this?

Producer.java

public class kafkaProducerRoute extends RouteBuilder{
@Override
public void configure() throws Exception {


   from("timer:time?")
            // loop over the route 10 times
            .setBody().constant("Hello, World!") // set the message body to "Hello, World!"
            .to("kafka:hello1?brokers=localhost:9092")  // send the message to a Kafka topic using the Kafka producer endpoint
            .log("Message is produced");



}

}

Consumer.java

    from("kafka:c1?brokers=localhost:9092")

            .log("Received message with offset: ${header." + KafkaConstants.OFFSET + "} - ${body}");

Solution

  • when the consumer group comes up again, does it come up with the same old group_id, if yes, then it should continue from where it left, if not, then this is normal as per the default setting for auto.offset.reset = latest will always start from the latest offset of the topic

    https://kafka.apache.org/documentation/#consumerconfigs_auto.offset.reset