Execute Python Script After PutFile (Apache - NiFi)

Question

I am trying to do something that I feel should be relatively simple, but I am struggling immensely to conceptualize how to execute this process.

At a high level I am simply unpacking a zip file into a directory. Once that is complete, I need to execute a python script.

The issue I am having is that I can't determine the best method to achieve this goal. I assume Notify/Wait will be the answer, but setting the release signal identifier and only releasing when all flowfiles have been placed, does not appear to be possible? I have also tried MergeContent using a like attribute, but I only want a single merged flowfile, and I cannot set the minimum # of entries = to ${fragment.count}. Please see below for any screenshots that may be beneficial. Thank you.

Answer 1

If you are using UnpackContent, all unpacked flowfiles get fragment identifier, index and count attributes - see the Writes Attributes section of the docs.

If you want to collect all of these into a single FlowFile, you can use MergeContent with a Merge Strategy of Defragment. This will take all of the FlowFiles created by the UnpackContent and merge them into a single FlowFile - it will group by the identifier and wait until it has seen every index up to the count - thus if 16 FlowFiles were produced, it will merge all 16 into 1.

You can use that single FlowFile to trigger your script.