Azure Data Factory, Copy Activity with "flatten hierarchy" Copy behavior

Question

I have a Data Factory data pipeline with a Copy Activity that has the 'Flatten Hierarchy' option. This option changes the file names in the destination. Is there a way to trace back the original file path from the auto-generated file name?

Answer 1

• There might be no way to retain path or filename while using flatten hierarchy. Even the file metadata (the following is the file metadata that I have used for demonstration and observed that it is not retained) is not being retained for the files that are copied using flatten hierarchy.

• You can flatten the files by yourself using loops in azure data factory if you know the depth of hierarchy. But this procedure might require a lot of for loop activities and this is not practically possible.
• Recursion is not supported in Azure data factory as the cycle would be detected and an error would be thrown.
• So, it might be not possible to reserve the information of file names and path using Azure data factory. As an alternative, you can use any programming language like Python with Databricks to flatten and preserve filename.