I have a copy activity in ADF to copy files from a blob storage container to another container.
Files are filtered using "Wildcard file path" like "Energy*.zip".
I want the pipeline to report failure if the number of copied files is zero.
I cannot use "Get metadat" as there are millions of files in the source and it results in the below error:
The length of execution output is over limit (around 4MB currently)
Is there any way to get count of copied files and report failure if it is zero?
Pipeline has only one Copy activity and there are no other activities.
You can use Fail activity to fail the pipeline. Upon the success of copy activity, check the number of files written equals to Zero or not in if activity and use fail activity inside True activities of if.
You can get the filesWritten count from the copy activity output.
Check this in if activity with below expression.
@equals(activity('Copy data1').output.filesWritten,0)
Inside True activities of if activity, take a Fail activity and give any custom message and custom error code like below.
When the files written count by copy activity is Zero, this will fail the pipeline with the above error message.