Search code examples
javaapache-kafkakafka-consumer-api

How should I properly create a new consumer group and reset the offset from all topics in kakfa CLI?

I have 1000+ topics in my kakfa, each topic contains 14 days of data, about 1.2 billion messages.

I created a new consumer group: groupA, and executed the command to reset the offset.

sh kafka-consumer-groups.sh --reset-offsets --execute --to-datetime "2023-05-03T00:00:00.000" --group groupA --bootstrap-server kfk-data- 001:9092,kfk-data-002:9092,kfk-data-003:9092 --all-topics

But it didn't work as I expected, I tried to use groupA pull data in Java, and it still consumes the starting data for each topics.

How should I properly create the group and reset the offset from all topics?

Solution

  • You don't need to use that command. In your consumer code, you could use offsetsForTime and seek function of consumer instance to go to the timestamp you want