Search code examples
hadoophiveooziehue

Oozie get a variable from a file

As an action in a workflow, Im writing the output of a Hive SELECT COUNT command to a text file in our hdfs.

I need to make a decision in the workflow after this, based on the value in the output.

I'm using HUE to construct this workflow.

What I need to know is:

  1. How to get this value as a variable, preferably not at the start of the workflow as it will be generated by the initial action.
  2. The syntax in the oozie decision for this variable.

I've tried my best to piece together other answers to similar questions but there's a few blank spaces I hope to fill in.

I'll be so grateful for any guidance! Thanks!

Solution

  • I would go for the Java path and capture-output. Oozie supports the Kerberos delegation using credentials (https://oozie.apache.org/docs/4.2.0/DG_ActionAuthentication.html) so you should not need to authenticate within Java; just use a standard JDBC connection