I am trying to customize the standard data ingest template available in kylo using a configuration file. I added the field name nifi.getfile.business_line1=Test in application.properties file available in kylo-services/conf directory. When i try to use nifi expressions in the processor ${business_line1} , it is throwing as business_line1 not found.
Or is there any other way to give customize the standard data ingest?
Can anyone please give a solution to this...
Using your example, Kylo will look for all GetFile processors when it creates a feed and replace the 'business_line1' property with the value 'Test'. However the GetFile processor does not allow custom properties and you'll get an error message similar to:
'business_line1' validated against 'Test' is invalid because 'business_line1' is not a supported property
There are a couple ways to solve this:
If you want to use NiFi expressions then you'll need to create an UpdateAttribute processor and add the 'business_line1' property there. Any subsequent processors would be able to access the value as ${business_line1}
If you want all GetFile processors to have the same value for a property, you can set the following property in Kylo's application.properties: nifi.getfile.file_filter=Test
The Kylo documentation goes into more detail about how to use variable substitution: http://kylo.readthedocs.io/en/latest/how-to-guides/ConfigurationProperties.html
Within NiFi you can use the NiFi Expression Language to access flow file attributes: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html