Search code examples
azuretriggersazure-data-factorydatatriggerazure-triggers

ADF Tumbling Window Trigger

I need help clarifying some concepts about trigger to pipeline relationship. So, I created a test pipeline where by activity name it is clear what it does Pipe line definition

  1. Sets the time now to @utcnow()
  2. Start time - trigger window start time (pipeline parameter) and
  3. End Time - trigger window end time. (pipeline parameter)
  4. Wait activity there is just to imitate pipeline running for longer than trigger window. (5 min = 300 seconds, my wait time is 310 seconds)

Here is a tumbling window trigger definition Tumbling window trigger

The help that i need is to understand this: When I go to the "Monitoring" page of the ADF, 1st in the "Pipeline runs" I see this Monitoring page screenshot 1

that tells me that the pipeline started at 4:10 and ended at 4:15 Then, I go the "Triggered runs" - and select "Tumbling window" tab

Tumbling window trigger

  • What is the relationship among the "Trigger time" and Window Start / End time ?
  • Is it ALWAYS the case that Trigger time is the same as the Window END time or it depends on the pipeline timing and is not guaranteed to be the same as Window End time? (Window start time) ?
  • How does the trigger time relates to the Pipeline start/end times?
Solution
    • What is the relationship among the “Trigger time” and Window Start / End time ?

    Here, in case of Tumbling window trigger, AFAIK, Trigger time always will be the Window End time.

    As per Documentation,

    Tumbling window trigger will create a back fill run(past run) if the trigger start time is in the past. It might be the reason for deciding the trigger time as the Window end time.

    For demo, these are my trigger runs and you can see Trigger time is same as Window End time for the first run.

    enter image description here

    But, when you look at the pipeline run, the first pipeline run started at trigger time.

    enter image description here

    The first trigger run is a back fill run for the specified interval.

    • Is it ALWAYS the case that Trigger time is the same as the Window END time or it depends on the pipeline timing and is not guaranteed to be the same as Window End time? (Window start time) ?

    Yes, it is always the Window End time in case of Tumbling window trigger.

    • How does the trigger time relates to the Pipeline start/end times?

    As mentioned in the above example, the Trigger run time is the Pipeline start time for that pipeline run.