Search code examples
apache-kafkaclickhouse

Force clickhouse to reset offset of kafka engine table

I have a kafka topic and a Created a MergeTree Table and a Kafka Table and a Materialized view to get data from topic . I need to consume message from earliest offset. My new consumer group offset set to latest message. But I can't find any setting to change it

Solution

  • As far as I know you cannot reset the offset for the consumer group using ClickHouse. However I think you can achieve the same result in two ways:

    1. Recreate the Kafka table with a different consumer group name using the kafka_group_name setting when creating the table.
    2. Set the offset using kafka-consumer-groups.sh. After quickly searching for how to do it, I think the following command should work after using the correct host and group name: kafka-consumer-groups.sh --bootstrap-server kafka-host:9092 --group my-group --reset-offsets --to-earliest --topic sales_topic --execute