If data source doesn't emit data for a while, does the watermark of Flink operator keep advancing?

Question

Just curious, if the data source doesn't emit data for a while, whether Flink operator still receive watermark or not? the Low Watermark in the image will keep advancing or stay still?

Answer 1

Whether the watermark will continue to advance when the source is completely idle depends on the WatermarkStrategy. Neither of the built-in strategies (i.e., BoundedOutOfOrderness or MonotonousTimestamps) will advance the watermark under these circumstances, but some folks use custom strategies that detect idleness and advance the watermark based on the passage of wall-clock time.

If some of the source partitions/splits/shards are idle and others are not, then this could lead to the watermark failing to advance. Some sources support a watermark strategy that implements a withIdleness option that overcomes this problem [1].

[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_timestamps_watermarks.html#dealing-with-idle-sources