Search code examples
azureazure-data-factoryazure-blob-storage

How to do copy of all the containers without explicitly mentioning in the array between two different storage accounts

https://stackoverflow.com/a/78351368/22054564

In my previous question-solution, i asked to change the names of the containers in the target storage account but now I do not want to change.

I have created

  1. Linked Services for source and target storage accounts.
  2. Integration Runtime created in Virtual Network as the storage accounts are in virtual network.

I don't know what are the next steps to copy all the containers as-is from source to destination. could anyone help me on this!

Solution

  • To achieve your desired requirement, you can follow below procedure:

    Create binary dataset with source linked services by leaving file path as empty as shown below:

    enter image description here

    Add Get meta data activity to the pipeline, select above data set for it with Child items as Field list shown below:

    enter image description here

    You will get list of all containers in the storage account as output of meta data activity as shown below:

    enter image description here

    Add for each activity on success of get meta data activity with @activity('Get Metadata1').output.childItems items.

    Create source and sink binary data sets with dataset parameter container_name for container name in file path as shown below:

    enter image description here

    Inside foreach add copy activity configure both source and sink with created datasets as shown below:

    Source:

    enter image description here

    Sink:

    enter image description here

    Debug the pipeline, copy all the containers as-is from source storage account to destination storage account as shown below:

    enter image description here