Search code examples
apache-nifi

In local I've folder which contains multiple csv files. using nifi how can I extact those files names and store it in other csv file?


I've multiple csv files in the folder and I want to extract all files name with extension and store those list of files names in separate csv file.

CSV files

I want output as this.

Expected Output

how to do that in apache nifi?

I tried listfile processor but it is only giving files with data not file names. I only need file names.


Solution

  • You are just a step away from getting your solution. The ListFile processor is what you need to achieve what you are looking for.

    You need to setup a Record Writer in your case you can use the CSVRecordSetWriter

    When a Record Writer is configured, a single FlowFile will be created that will contain a Record for each file in the directory, instead of a separate FlowFile per file.

    Listfile Properties

    Output Flowfile

    From here on you can use QueryRecord processor and use a simple SQL query to get just the columns that you need for further processing.

    QueryRecord Properties

    Output