I am messing aroung with Kafka Streams handled by the K8s. It goes more or less fine so far, yet weird behaviour is observed on the test environment:
[Consumer clientId=dbe-livestream-kafka-streams-77185a88-71a7-40cd-8774-aeecc04054e1-StreamThread-1-consumer, groupId=dbe-livestream-kafka-streams] We received an assignment [_livestream.dbe.tradingcore.sporteventmappings-table-0, _livestream.dbe.tradingcore.sporteventmappings-table-2, _livestream.dbe.tradingcore.sporteventmappings-table-4, _livestream.dbe.tradingcore.sporteventmappings-table-6, livestream.dbe.tennis.results-table-0, livestream.dbe.tennis.results-table-2, livestream.dbe.tennis.results-table-4, livestream.dbe.tennis.results-table-6, _livestream.dbe.betmanager.sporteventmappings-table-0, _livestream.dbe.betmanager.sporteventmappings-table-2, _livestream.dbe.betmanager.sporteventmappings-table-4, _livestream.dbe.betmanager.sporteventmappings-table-6] that doesn't match our current subscription Subscribe(_livestream.dbe.betmanager.sporteventmappings-table|_livestream.dbe.trading_states|_livestream.dbe.tradingcore.sporteventmappings-table|livestream.dbe.tennis.markets|livestream.dbe.tennis.markets-table); it is likely that the subscription has changed since we joined the group. Will try re-join the group with current subscription
As far as I understand, internal state somehow got broken, and Stream's source of truth conflicts with the broker/zookeeper's one. This behaviour never terminates: I just let it hang for few days beingh busy with another stuff, and still it's the, reported at the WARN level. More than that: no ERRORs were reported for this time.
I did not change nothing; did not deploy new instances; did not manipulate Kafka brokers in any way that might affect abovementioned Kafka Streams app. Any ideas what's wrong?
The error message itself indicates, that something is wrong with your subscription. This may happen if you have two Kafka Streams instances using the same application.id, but both don't subscribe to the exact same topics.
In your case, the subscription does not contain livestream.dbe.tennis.results-table but corresponding partitions are assigned.
Note, that Kafka Streams requires that all instances with the same application.id are required to execute the exact same Topology and thus subscribe to the exact same topics.