We are using kafka cluster with 3 kafka nodes with kafka version 0.10
From the kafka logs we get a very strange message as the following
Beginning offset 1432 is after the ending offset 1298 for topic data.out.cars
note - we have 110 partition for this topic
How it can be that begin offset is 1432 while ending is much less as 1298
What could be wrong in kafka configuration ?
Can we avoid this problem by tune some kafka parameters?
What I suspect due to retention data got purged and offset got jumped to a new beginning. You can validate by checking the topic offset from-beginning position. If the message got truncated it will get started from a new position that will appear on-topic partition detail.
bin/kafka-console-consumer.sh --zookeeper <zk_host>:2181 --topic test --from-beginning
If you are using a new API then use below
bin/kafka-console-consumer.sh --bootstrap-server <broker_host>:6667 --topic test --from-beginnin