Issue with Filtering data With Filter Activity in Azure Data Factory

Question

I am trying to filter the following folders using filter activity and select only the latest date from entire blob Container.

Folder Names example: 2020-04-21, 2021-08-11, 2020-01-07

I need to select 2021-08-11

I am using Get metadata (ChildItems) activity to get folder names and using following logic to filter them in filter activity.

@greater(max(array(int(replace(item().name, '-', '')))),0)

But after filter I am getting all folders as they are. I know my logic is scuffed, but I am beginner in ADF

Answer 1

You can do first get Metadata to get full list of folders, use ForEach to find the latest folder and assign the folder name to a variable, finally do the copy data using the variable, which now stores your latest folder name.

Parent pipeline:

output of GetMetadata:

Dynamic content used in ForEach: @activity('Get Metadata1').output.childItems

The forEach contains only 1 if condition activity:

The expression used is imilar to yours: @greater(int(replace(item().name, '-', '')), int(replace(variables('latestFolder'),'-','')))

Inside the true case, there is only the variable assignment:

finally use the variable, which now contains the latest folder's folder name, in the copy activity: