Azure Data Factory Copy Files

Question

I want to copy files from one folder to another folder in data lake using ADF pipelines. Ex : a/b/c/d. TO a/b Here a,b,c,d are folders here I don't want to copy c,d folders .I have to copy the files inside those folders to 'b' folder.

I created a pipeline using Get Metadata , For each and in For Each I used copy activity.But here I am able to copy files with folder itself .I'm failing to remove folders.

Answer 1

I reproduced your scenario follow the below steps:

• In my demo container I have nested folders like a/b/c/d under d folder I have 3 files as below.

• To copy files from folder to folder I took Get metadata activity to get list of files from folder. Dataset for Get Metadata: Get Metadata settings:

• Then I took for-each activity and passed the output of Get Metadata activities output to it.

    @activity('Get Metadata1').output.childItems

• Then created copy activity inside for each activity and created source dataset with filename parameter In file name gave dynamic value as @dataset().filename In copy activity source gave dynamic value for dataset property filename as @item().name Now created sink dataset with a/b directories only and passed it to sink

Output

• files copied under b folder without coping c and d folder