Search code examples
azure-data-factory

Why ADF Monitoring does not show the status of the last activity ran by pipeline

I have an overall design like this picture below. When the Condition is FALSE it correctly goes to the False part, runs the FAIL ACTIVITY I have in there and correctly takes the "fail route" of the IF activity and correctly and successfully also sets the value of the variable to NO. So far all good and what I expected. However when I go to Monitor the pipeline , it shows a FAIL . My expectation is that it should show a SUCCESS because the last activity that pipeline ran was that Set Variable and it was successful. So why it shows Fail in the monitoring section ? And also how can I get it to show success ? enter image description here

Solution

  • Here, the if activity failed because the inner activity failed, and you handled it by using a set variable on failure of it.

    But the reason for the pipeline failure is the Fail activity. In false activities, the Fail activity will give a failure and you didn't handle it.

    enter image description here

    enter image description here

    A pipeline fails when the last(leaf) activities fail and here, the Fail activity is a leaf activity.

    To solve this, you can embed the failure activities in False activities but as you have more if activities, this won't work.

    The following can be a workaround for this scenario but this method is quite complex if there are more if activities in your pipeline.

    Use the Do If Skip Else block from the documentation inside the False activities.

    As nested if not supported in ADF, use the set variable as if like below.

    enter image description here

    @if(<your condition>,'Yes',1)

    Here, I am using the Success and failure of the set variable activity as True and false of if.

    If your condition is true, then it will set 'Yes' string value to the string type variable samp and if it is false, it will set the 1 integer value to samp which will generate the error.

    First use an if activity and inside the False activities use this logic as nested if.

    This won't fail the pipeline on the failure flow.

    NOTE: If you have lot if activities, the above method may get complex, so it's better to use child pipelines in that case as suggested in comments.