Kafka 0.10.0.1 adds the ability to reset a Kafka Streams app using a script called kafka-streams-application-reset.sh
Confluent has some good documentation on this script here.
Unfortunately, running the script locally results in an error:
$ bin/kafka-streams-application-reset --application-id my-streams-app \
--input-topics my-input-topic \
--intermediate-topics rekeyed-topic \
--bootstrap-servers brokerHost:9092 \
--zookeeper zookeeperHost:2181
# output
ERROR: Request GROUP_COORDINATOR failed on brokers List(...)
Also, the following line appears in the broker logs:
kafka.network.InvalidRequestException: Error getting request for apiKey: 3 and apiVersion: 2
From what I can tell, it looks like the client is making an invalid request, but I'm unsure why this happening. Our brokers are still on 0.9.0 so I'm not sure if that's the issue, but it appears to be a meta data request based on the apiKey value in the broker log.
Could someone please let me know why this might be happening and how it can be resolved?
Kafka Streams is based on version 0.10 and does not work for 0.9.
The same is true for bin/kafka-streams-application-reset tool. It uses a 0.10 KafkaClient internally and requires a 0.10 broker.
In general, only Kafka brokers but not Kafka clients are backward compatible, i.e., brokers do work with older clients but clients do not work with older brokers. This might improve in future released so. See KIP-97: Improved Kafka Client RPC Compatibility Policy
The error message -- even if not very descriptive and hard to understand -- is telling you that you try to connect to a broker that does not understand the new client.