Is it possible to delay a window trigger without changing window-end and window-start times? Is the only solution developing a custom EventTimeTrigger?
I use 1 hour window period for processing different type of messages. As they trigger at the same time, I encounter a lot of resource problems. I just want to put some random delays before process() is executing, without changing the content of windows.
If I need to develop a custom EventTimeTrigger, is it possible to delay the fire , without changing the period start-end times and content?
Flink's time windows take an optional WindowStagger parameter (it's an enum) that can be used to avoid the thundering herd problem that arises from have all of the partitions fire their windows at the same time.
The default staggering policy is ALIGNED, meaning all panes fire at the same time across all partitions. The other options are RANDOM (hopefully obvious), and NATURAL, meaning the panes are staggered based on when each parallel operator received the first event.
Take a look at the docs for TumblingEventTimeWindows of(Time size, Time offset, WindowStagger windowStagger).