If the Kafka broker is configured with Log-Append Time
, will the broker overwrite the message timestamp regardless of the timestamp set by the producer/stream?
From streams-time - Documentation, I get the impression that CreateTime
is somewhat unreliable - the timestamp is sometimes only forwarded (e.g. when using context.forward()
) and sometimes overwritten (e.g. when using puntuate()
). I am therefore wondering if the Log-Append Time
takes precedence when setting the message timestamp (i.e. overwrites the timestamp set by the stream)?
Yes, LogAppendTime
takes precedence, meaning if the topic is configured to use LogAppendTime
, the timestamp set by the Streams application (no matter how it is generated) will be ignored and overwritten with the log append time by the broker when the output record is produced.
Furthermore, I don't think CreateTime
is unreliable in the way you think. punctuate()
doesn't overwrite an existing timestamp - it is used to generate a fresh record at a specific point in time, so there is no previous CreateTime
available - Streams must pick a new timestamp.