Search code examples
apache-nifi

NiFi : Prepend (orAppend) a line number to all records in a flowfile

I am using NiFi 1.9.2

I am reading a text file which happens to be a csv file. I have the Contents of the file in the Contents of a flowFile.

Contents are

a,b,c
d,e,f
g,h,i

I want to prepend a line number to all records in the flowfile and get

1,a,b,c
2,d,e,f
3,g,h,i

each time I feed a file through this processor

I can achieve something close by using the ReplaceText processor with Properties as follows:

Search Value : (?m)(^.*$)
Replacement Value : ${nextInt()},$1

But because nextInt() persists it's value over the lifetime of the running NiFi instance I get

    0,a,b,c
    1,d,e,f
    2,g,h,i
for 1st execution

    3,a,b,c
    4,d,e,f
    5,g,h,i
for the next execution etc

Additionally, from the NiFi expression-language-guide, the "counter is shared across all NiFi components, so calling this function multiple times from one Processor will not guarantee sequential values within the context of a particular Processor."

  1. Is there a way to ensure the line numbers always start at 0 for each execution of this processor for the lifetime of the NiFi instance, and are always sequential?

  2. What the range of the counter?

  3. Can I get the counter to start at 1?

Solution

  • You can split the content to several lines then use fragment.index to prepent the counter to the lines. After that you can merge them again.

    The Flow: enter image description here

    GenerateFlowFile: enter image description here

    SplitText: enter image description here

    ReplaceText: enter image description here

    MergeContent: enter image description here Don't forget to add a new line (Shift+Enter) to Demarcator attribute.

    Result: enter image description here

    You can use ${Fragment.index:minus(1)} if you want to count from zero.