Search code examples
javamavenapache-nifi

Cannot resolve org.apache.nifi:nifi-standard-services-api-nar - How to use LookupService inside CustomProcessor

I'm implementing Custom NiFi Processor to decode some files. During the decode process I need to lookup external CSV record for certain data mapping. Note the this decoding process is complex so it has to be a custom processor. I know I can setup external CSVRecordLookupService and then within my CustomController I can define a property descriptor as below

    public static final PropertyDescriptor CLIENT_LOOKUP_SERVICE =
            new PropertyDescriptor.Builder()
                    .name("Client CSV Lookup service")
                    .identifiesControllerService(LookupService.class)
                    .required(true)
                    .build();

My first problem is How to refer LookupService.class, which maven Jar package should I use. After bit of investigation I found this https://mvnrepository.com/artifact/org.apache.nifi/nifi-standard-services-api-nar/1.13.0. So I added it to my processors pom.xml as below

        <!-- https://mvnrepository.com/artifact/org.apache.nifi/nifi-standard-services-api-nar -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-standard-services-api-nar</artifactId>
            <version>1.13.0</version>
        </dependency>

Now I can compile my class and LookupService is visible to the code. However I cannot build the nar or run the maven as it gives Cannot resolve org.apache.nifi:nifi-standard-services-api-nar:1.13.0 error

in the IDEA it also shows the error

enter image description here

How to solve this issue and use org.apache.nifi.lookup.LookupService in my custom processor.

I believe it probably because it comes as nar instead of jar. But my question is how do I use LookupService at compile time only and use the NiFi provided classes at run time by solving this issue.

UPDATE

I'd like to provide an update on this, but the issue is not fully resolved.

I found a jar in maven repo which has the LookupService

This together with org.apache.nifi.serialization.record.Record I can compile and run the maven successfully. 

According to maven repo dependency both these should be declared with the scope provided. So I did that.

Now the problem is I cannot package the nar. It doesn't like the I excluded the LookupService class. even if I remove provided scope it doesn't get included as well.

[INFO] Generating documentation for NiFi extensions in the NAR...
[INFO] Found a dependency on version 1.13.0 of NiFi API
[ERROR] Could not generate extensions' documentation
java.lang.NoClassDefFoundError: org/apache/nifi/lookup/LookupService

...
...

[ERROR] Failed to execute goal org.apache.nifi:nifi-nar-maven-plugin:1.3.1:nar (default-nar) on project nifi-decoder-processors-nar: Execution default-nar of goal org.apache.nifi:nifi-nar-maven-plugin:1.3.1:nar failed: A required class was missing while executing org.apache.nifi:nifi-nar-maven-plugin:1.3.1:nar: org/apache/nifi/lookup/LookupService
[ERROR] -----------------------------------------------------
[ERROR] realm =    extension>org.apache.nifi:nifi-nar-maven-plugin:1.3.1

So how do we use this LookupService in our custom processors?

Solution

  • in order to make use of it in runtime you must add below dependency in nifi-decoder-processors-nar's pom file. type is nar

        <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-standard-services-api-nar</artifactId>
        <version>1.13.0</version>
        <type>nar</type>
    </dependency>

    as for compiler time add below in nifi-decoder-processors-processors ' pom file (I found it in Nar's pom.xml in repository, you may think NAR as collection of dependency)

        <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-lookup-service-api</artifactId>
        <version>1.13.0</version>
        <scope>provided</scope>
    </dependency>

    see this post