Search code examples
apache-kafkakafka-producer-api

If the producer performs a retry, will the two messages have the same timestamp?


At-least-once semantics: if the producer receives an acknowledgement (ack) from the Kafka broker and acks=all, it means that the message has been written exactly once to the Kafka topic. However, if a producer ack times out or receives an error, it might retry sending the message assuming that the message was not written to the Kafka topic. If the broker had failed right before it sent the ack but after the message was successfully written to the Kafka topic, this retry leads to the message being written twice and hence delivered more than once to the end consumer.

I know that the timestamp is set based on the time when the message is sent from the producer. If the producer performs a retry, will the two messages have the same timestamp?


Solution

  • By default, timestamps are set at the record creation. So, yes, if the producer performs a retry, the duplicates have the same timestamp.

    If you set message.timestamp.type to LogAppendTime, then duplicates will have different timestamp, each record's timestamp will be the effective time the broker appended the it to the log.