Search code examples
apache-kafka-streams

KafkaStreams handling late events when using windowing


Question: If an event arrives after the window has closed, then, how do we re-direct it to another topic for handling the correction ?

Context:

  • We use tumbling windows
  • We use events source creation time(event-time) for defining windows

thanks


Solution

  • Currently, there is no API to do that. Late events are dropped and you cannot get a hand on them easily.

    What you could do is, to have an upstream operator (like a transform()) before the window, the compares the record timestamp to the current "stream time" (you would need to track "stream time" manually within the operator) -- this should help you to detect if the downstream window will drop the record as late and react to it accordingly (for example using a branch() after transform() and before groupByKey().windonwedBy().