Azure Data Factory - Manual Trigger Execution Time
I've been checking some of the results provided by ADF. I'm new with this tool, and I would like to understand more about the execution time that I see after running the ETL with the trigger or the debug option.
My question is, why the difference at the end of the process? Debug is almost 25% of the other. What happens when I execute an ETL with the trigger option that increases the processing time comparing with the debug.
Yes, if you put the dataflow in a pipeline and start data flow debug in preview, then you will see something like this:
Trigger pipeline seems take more times than debug it, but this does not mean that debug has better performance. You just dont count the start time of data flow debug module:
The time that the trigger seems take more time than debug is because the dataflow debug module. Please have a look of this doc:
When you trigger a pipeline, you need to initialize the Integration Runtime. This will take a lot of time. Why you debug the pipeline seems take less time is because you have already start the dataflow debug module. You can try to close this debug module, and then you will find debug also take about 5~6 minutes:
