numLateRecordsDropped: What does it mean for operators

Question

There are multiple tasks here. One of the tasks is BookingInfoWithFraudAndDefaultAndMainSP -> TSAndWMBookingWithSPObjects. Lets call it task-1. At task-1, I assign a timestamp and generate watermark, I am using BoundedOutOfOrdernessTimestampExtractor with maxOutOfOrderness equal to 2min.

The next operator is where I window the data and do some aggregations on top it which are then sinked to Kafka. Lets call this chained task of Aggregating and Sinking, Task-2.

numLateRecordsDropped: Looking at this metric which tells the The number of records this operator/task has dropped due to arriving late.

Question: When I window elements, i have assigned 0 allowed Lateness. So it could have dropped some elements. But when I look at the metrics, since window is not an operator, there is no metric which can tell how many elements are being dropped by windows.

When I look at task-2 metrics, it shows a count for numLateRecordsDropped. What does it mean. How can Window aggregation task drop records. Or since it is aggregating windows, the count basically is the number of records dropped by windows.

Answer 1

The Window operator is the only place where Flink uses numLateRecordsDropped (and furthermore, the window aggregation function runs in the window operator), so yes, the count is the number of records dropped by the window.