Copy data Activity Azure synapse

Question

Hi All, we are looking at automating a copy data activity using a for each loop. The Metadata1 gets the list of the file names, from here we use Bronze/@activity('Get Metadata1').output.childItems / * to get the filepath to iterate over.

But it seems to error unfortunately, any ideas? I feel it is something strange with the naming convention

Answer 1

The Metadata1 gets the list of the file names, from here we use Bronze/@activity('Get Metadata1').output.childItems / * to get the filepath to iterate over.

The Get meta data activity will give the child items like below.

"childItems": [
        {
            "name": "LOWfixthree.csv",
            "type": "File"
        },
        {
            "name": "Table2.csv",
            "type": "File"
        },
        {
            "name": "sample_blank.csv",
            "type": "File"
        }
    ]

You can see it is an array and you should not use that in the filepath like that.

To achieve your requirement, give the child items array @activity('Get Metadata1').output.childItems to the for-each activity expression.

In the copy activity inside the for-each, use this array iterations for giving the current file name. Give the @item().name in the source wild card path like below.

If you have the data inside any folder of your bronze container, you can give that folder path as well in the above.

Give your target location in your sink dataset and debug the pipeline. The pipeline will copy each file from the source to target in each iteration of the Get meta data child items array.