I had a kafka consumer for a specific topic. Because of reasons I changed the consumer. Instead of consuming the message now I have a kafka-stream which handles the topic and produces new messages to different topics. This works fine.
My problem is I looked at the kafka messages with an UI and found out that kafka now thinks the old consumer is thousands of messages behind. I would like to tell kafka that this consumer now is gone or should I just ignore this? I cannot remove the complete consumer group because I have some consumers with the same group on other topics.
you can use the kafka-consumer-groups tool and specify the group and topic to be deleted. It has a delete option.
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --delete --group my-group --topic my-topic
If you are not sure you can use the dry-run option before executing to check the output.
You can find further documentation here.