Search code examples
azure-data-factoryderived-column

Getting Data Lake Metadata in Azure Data Factory's Data Flow

I want to add the timestamp of copying parquet files to my dataframe in data flow as a derived column.
In source module I can filter parquet files by last modified which makes me think that it should be possible to access files' metadata including copied timestamp through derived column transformations, but I couldn't find anything for it in Microsoft documentation.
enter image description here

Solution

  • There is no function can get the last modified time in the data flow expression.

    As a workaround, you can create a Get Metadata activity to get that and then pass it's value to a parameter in your data flow.

    enter image description here

    The expression:@activity('Get Metadata1').output.lastModified

    enter image description here