Search code examples
pipelineazure-synapseazure-synapse-analyticsazure-synapse-pipeline

Azure Synapse Pipeline Get Metadata Last Modified Date on file instead of on folder

I am trying to get the Last Modified date on the files (CSV linked service) in my repository using Get Metadata activity.

Field List

However, I am getting the same date/time for all of the files... the LastModified date on the folder instead of the individual files inside.

same LastModifiedDate output example

My design pattern is to Get the list of files and then iterate in a For Each over each item.

Synapse Pipeline design pattern

I am able to get the Filenames with

@activity('Get Metadata').output.childItems

I just don't see a LastModified date in the attributes of the Jason.

This code needs to go a level deeper:

@activity('Get Metadata').output.lastModified

What is the syntax I am missing to access each of the file properties?

Solution

  • I could not get Pratik's solution to work.

    I had to do the following:

    A) Pass childItems to ForEach

    @activity('Get Metadata1').output.childItems

    enter image description here

    B) On the ForEach GetMetadata2, use a copy of the dataset. Add item.name to the filename value and select the Last Modified in the Field List:

    @item().name

    On the ForEach GetMetadata2, use a copy of the dataset. Add item.name

    C) Open the copy of the dataset and add to the filename

    @dataset().filename

    Open the copy of the dataset and add to the filename