Search code examples
marklogicapache-nifimarklogic-dhf

Run a MarkLogic Data Hub 5 Flow from Apache Nifi


I succeed in using the PutMarkLogic processor in Nifi to insert JSON documents into MarkLogic. This creates plain JSON documents. How can I add a data hub envelope during ingestion? How can I start a data hub 5 flow during ingestion?

According to the MarkLogic Nifi documentation at https://marklogic.github.io/nifi/run-data-hub-input-flow this can be done in the PutMarkLogic processor with the property "Server transform" = "ml:sjsInputFlow" and the custom properties trans:flow-name and trans:entity-name. But this results in an "Invalid coercion" error in MarkLogic:

2020-03-10 17:09:44.967 Notice: XDMP-AS: $src as node() -- Invalid coercion: null as node()
2020-03-10 17:09:44.967 Notice:+in /data-hub/4/impl/flow-lib.sjs, at 56:16, in getFlow() [javascript]
2020-03-10 17:09:44.967 Notice:+in /marklogic.rest.transform/mlSjsInputFlow/assets/transform.sjs, at 25:21, in transform() [javascript]
...

In the stacktrace data-hub 4 is mentioned. So maybe the MarkLogic Nifi documentation is outdated?


Solution

  • That's for a DHF 4 flow. Use "mlRunIngest" as the transform name and "trans:flow-name" to specify the flow name and "trans:step" to specify the step number.