Search code examples
docker-composeapache-nifi

How to let nifi in docker can access my custom processor nar file?

Environment:

  • OS: Ubuntu Server 20.04 LTS
  • Docker: 23.0
  • NIFI: 1.20.0

I use docker-compose.yml to build a nifi in docker.

docker-compose.yml:

version: "2.16.0"
services:
  nifi-1:
    image: apache/nifi:1.20.0
    container_name: nifi-1
    hostname: nifi-1
    ports:
      - 9080:9080
    environment:
      - NIFI_REMOTE_INPUT_SECURE=false
      - NIFI_WEB_HTTPS_PORT=
      - NIFI_WEB_HTTPS_PORT=
      - NIFI_WEB_HTTP_HOST=0.0.0.0
      - NIFI_WEB_HTTP_PORT=9080
      - NIFI_SECURITY_KEYSTORE=
      - NIFI_SECURITY_KEYSTORE_TYPE=
      - NIFI_SECURITY_TRUSTSTORE=
      - NIFI_SECURITY_TRUSTSTORE_TYPE=
    volumes:
      - /opt/nifi-3rd-lib:/opt/nifi-3rd-lib

I have developed my custom processors and packed them into a nar file (say foo.nar) and put it in /opt/nifi-3rd-lib in host. I want my nifi in docker can use foo.nar. What should I do ?

My original thought is:

  1. put foo.jar in /opt/nifi-3rd-lib in host
  2. Add (or change) classpath for nifi in docker-compose.yml (but I don't know how to make it)

Therefore, the second thought is:

  1. Start docker-compose (but don't start nifi yet)
  2. Ask docker-compose to copy foo.nar in host into nifi-current/lib in container (I don't know how to do it neither)
  3. start nifi

So, could anyone give instructions for either the above thoughts ? appreciate

Solution

  • https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#autoloading-processors

    the nifi.nar.library.autoload.directory property must be configured to point at the desired directory. By default, this points at ./extensions.

    So, just map local folder with all your custom nar files into ${NIFI_HOME}/extensions folder

    Normally NIFI_HOME=/opt/nifi/nifi-current but please doublecheck nifi location in your container

        volumes:
      - /opt/local-nar/:/opt/nifi/nifi-current

    official Dockerfile located here: https://github.com/apache/nifi/blob/main/nifi-docker/dockerhub/Dockerfile - it could give you better understanding how you could change the behavior of dockerized nifi.

    As alternative option you could change an entrypoint of image using docker-compose.yml and run own script just before starting nifi. like this:

    version: "2.16.0"
services:
  nifi-1:
    ...
    volumes:
      - /opt/nifi-3rd-lib:/opt/nifi-3rd-lib
    entrypoint: /opt/nifi-3rd-lib/pre_start.sh && ../scripts/start.sh

    and in pre_start.sh you can copy all required files into ${NIFI_HOME}/extensions. NIFI_HOME variable should be available in script..