Search code examples
azure-data-factory

Azure data factory - Concurrence problem in the child pipeline


i have a parent and child pipelines. Parent pipeline triggers the child pipeline inside the foreach activity. I have totally 6 items in the array so foreach triggers the child pipeline 6 times. I made foreach to trigger child pipeline parallel by unchecking the sequential check box. I have also have given 6 concurrence to the child pipeline. Currently 6 instances of the child pipeline is running at the same and ends at the same time. I want the child pipeline run parallel but i want start each instance by 30 or 40 secs delay or end by 30-40 secs difference. I have next activity which check the flag.

I added the delay at the end of the child pipeline but the wait exactly ends on the same time for 6 instances. I also tried adding set counter and increase the counter but two times counters are intialized with the same value as the 2 instances of the pipeline ends at the same time.


Solution

  • You many use rand function to generate random number in between range and use that number in wait activity so that you will have gap in execution of child pipelines.