I have a simple topology that does some windowing on a topic. I haven't made any changes to the topology nor do I set the partition size on any internal topic.
The only thing I did change was the number of stream threads (NUM_STREAM_THREADS_CONFIG) from 10 to 50. Setting it back to 10 did not fix this.
2021-10-07 22:48:24.452 ERROR : o.a.k.s.p.i.InternalTopicManager: stream-thread [main]
Existing internal topic my-app-KTABLE-SUPPRESS-STATE-STORE-0000000015-changelog
has invalid partitions: expected: 10 ; actual: 1.
Use 'kafka.tools.StreamsResetter' tool to clean up invalid topics before processing.
From previous answers it appears the way to fix this is to reset streams for the entire application (causing reprocessing from the beginning). This isn't ideal. I may try just deleting all the topics relating to this app and starting over.
My question - I would like to understand why this happened. It seems like a pretty serious error without any visible cause.
nor do I set the partition size on any internal topic
That's why you're getting an error. Increasing threads requires a higher partition count, since you can't have more consumer threads * instances than total partitions. The default is only one partition.
You also cannot decrease a topics partition count, without deleting the topic and recreating it, which is what the reset command does; it doesn't delete "user topics" that still have data in them, only resets the group id and deletes the internal topics
You don't need to reset the app since you can use kafka-topics and kafka-reassign-partitions CLI tools to add partitions manually