Search code examples
azurehttpazure-data-factory

How do I create an Azure Data Factory ingest task that can request **multiple** Http urls?


I am using Azure Data Factory for the first time.

I have a collection of urls that I want to GET using the copyData task. They all follow the pattern** .../video_1; .../video_2; ../video_3** and so on.

I have tried using dynamic parameters, but I could not understand it and couldn't get it to work.


Solution

  • create an array parameter containing the urls and loop through then using the Foreach activity. You have the option to handle it sequentially or as a batch. Inside the foreach activity you put in your further activites like a copydata or smth else, that uses the array1,array2 and so on.

    For the Numbers you could use the expression builder function named range(1,20) to have the names like video_1 ....video_20