Kafka Streams Application consumer offset getting reset to Earliest on restart

Question

We have a Kafka Streams application that is experiencing weird behavior. Randomly when the job is killed and restarted, the consumer group is getting its offsets reset to earliest and all the old records are getting reprocessed.

Is there anything specific that needs to be done and we're missing that?

"buffered.records.per.partition": 512000, 
"cache.max.bytes.buffering": 134217728, 
"commit.interval.ms": 30000, 
"retries": 2147483647, 
"acks": "all", 
"processing.guarantee": "exactly_once", 
"compression.type": "snappy", 
"auto.offset.reset": "latest"

Answer 1

Well, you have set "auto.offset.reset": "latest", so clearly that property isn't getting applied.

To guarantee at-least-once processing semantics and turn off auto commits, Kafka Streams overrides enable.auto.commit consumer config value to false. Consumers will only commit explicitly via commitSync calls when the Kafka Streams library or a user decides to commit the current processing state

- docs

Therefore, it seems you have not performed a terminal action on any stream item such that it would make a synced commit.