Let's say I have two consumers - C1 and C2 in one consumer group. There are 10 messages on a topic before both C1 and C2 came to life. C1 reads the first 5 messages and goes down while C2 reads messages9-10. C1 comes back after some time. Now, how does Kafka identify that consumer that just came up is indeed C1 and it allows it to read from its last committed offest, did it save some information when C1 was registered in the group earlier so that it can uniquely identify C1 when it came back up again?
Offsets are not bind to actual Kafka Consumer instances but to consumer groups instead.
More precisely, the committed offsets for each topic/partition per consumer group are stored in __consumer_offsets
topic (this is a compacted topic so that only the latest offsets are kept).
In your case, once consumer C1 is up and running again, it will normally start consuming data from the latest committed offset of the consumer group in which the consumer belongs to (this depends on the configuration though - you can force it to start from the beginning). In your case, C1 will start consuming messages from the 11th message onwards.