I am running a data pipeline such that I read data from sql db into kafka topic through jdbc connect I sink this data in Elasticsearch using kafka sink connector for ES
I have a need to reset this pipeline. To that end I want to reset the consumer group that is listening on the jdbc connect topics. So I run the following command
kafka-consumer-groups --reset-offsets --to-earliest --all-topics --execute --group mygroup --bootstrap-server myserver:9092
But I get this error
Error: Assignments can only be reset if the group 'mygroup' is inactive, but the current state is Stable.
I stopped the connect to make the group inactive but that didnt work. So my question is, how do I make my group Inactive
--reset-offsets needs to check if a consumer is currently active before resetting the offsets. Stable indicates that the consumer is currently active and therefore offsets cannot be shifted back.
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group mygroup --describe
should give any active consumer groups (if any). The output will help you identify which applications are currently active. Once the apps that keep consumers active are killed, you will be able to reset the offsets.
Alternatively, if you are using old consumers you can delete a consumer group(s) by running
bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --group mygroup --delete
and finally reset the offsets.