Search code examples
sql-serverazure-data-factory

Converting Multiple Fixed Width Files to CSV at a time in ADF


I have three (.txt) files in a container. I need to convert all these three files at a time to CSV.

enter image description here

I have converted Fixed Width to CSV only one file at a time. enter image description here But my requirement is to convert multiple files at a time which are available inside a container. Could anyone please helps me with a solution.Requirement is, daily we get multiple fixed width text files and we need to convert them to CSV and load them in Tables


Solution

  • In a pipeline, use a metadata activity to get a list of files in the folder. Pass that list to a foreach loop. Inside the loop, run the dataflow you show above, and pass to it the parameters of file source and destination.