apache nifi total execution time

Question

I would like to know how to find out/capture the total execution time for any flow in Apache Nifi. Is there a way to do that and add it to the attributes list so that same can be shared over the PutEmail?

Answer 1

I believe you would have to add a custom timestamp property when the flowfile is received, do the bulk of your processing, then calculate an elapsed time attribute to include in the email. You can do this with two UpdateAttribute processors.

1. UpdateAttribute, received = ${now():toNumber()}

2. (do processing)

3. UpdateAttribute, elapsed = ${now():toNumber():minus(${received}):format("HH:mm:ss")}

This will format the elapsed time like "00:04:16" (4 minutes, 16 seconds). You can use this as ${elapsed} in your PutEmail message content.

But this is a bit ugly and only gives an approximation of the processing time. The NiFi provenance system maintains "Lineage Duration", which describes the elapsed time since the file entered NiFi. This is a far more authoritative number. But I do not believe you can query lineage duration from expression language. You would have to separately query and analyze the provenance data.