Search code examples
pythonjavabigdataapache-nifi

Automation of downloading NiFi flows and uploading them to another cluster

I am relatively new to NiFi and I was wondering if it is possible to automate the process of downloading some NiFi templates and then uploading them to another cluster.

More precisely, I created a local flow, which I downloaded and uploaded to a server where it runs continuously. I would like to work on the local flow, and when I make changes, to download and upload automatically on the flow running on the server.

Is there any possibility to do this?

Thanks:)

I did everything manually, I wish it could be automated.

Solution

  • @Calin

    Everything you describe is possible programatically using the NIFI API.

    Some things to consider:

    1. The nifi api is available from command line or within a nifi flow (invokeHttp). So you can make a nifi flow that does this. DOCS
    2. Check out Nifi Registry as a mechanism to version and move flows between enviornments.
    3. Check out JSON Flow Definition Files vs NiFi XML Templates (these are going away) DOCS
    4. Check out this flow I built that demos how to use the API within a nifi flow itself. This particular flow is used to convert XML to JSON, but it has the framework needed to execute the api calls within nifi itself.