Search code examples
oozieoozie-coordinator

Is it possible to log a message from an Oozie Workflow without killing it

I've got a somewhat complex Oozie work flow and I'd like to be able to see some of the properties at various points in the flow. The obvious solution is to simply write a message to the logs.

I've gone through the official documentation, and it appears that the only action that supports logging is the Kill action!

Is there anyway to log a set of properties without killing the workflow?

Update (to provide more details)

So imagine my (simplified) workflow looks something like:

<workflow-app name="myWorkflow" xmlns="uri:oozie:workflow:0.5">
  <start to="action1"/>

  <kill name="Kill">
    <message>Failed: error ${myField}</message>
  </kill>

  <action name='action1'>
    <java>
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <configuration>
           <property>
                <name>mapred.job.queue.name</name>
                <value>default</value>
            </property>
        </configuration>
        <main-class>my.Class</main-class>
        <arg>${myArgument}</arg>
        <capture-output/>
    </java>
    <ok to="action2" />
    <error to="Kill" />
  </action>

  <action name='action2'>
    <ok to="End" />
    <error to="Kill" />
  </action>

  <end name="End"/>
</workflow-app>

Inside the workflow I've got fields called ${myField}, ${myArgument}, ${jobTracker} and ${nameNode}. Some of these are properties set by the launching process others will be set by the Java class I call from 'action1'. In-between executing 'action1' and 'action2' I'd like log the values of these fields. I don't much care where the values are logged to, I just need to be able to see what the values are.

I do NOT want to call my 'kill' action to do the logging because it's important that 'action2' is run.

Solution

  • One way I could see is you can put a shell script at the end of your workflow with the values to be logged as parameters to the shell script. Inside the shell script write the values of those parameters in a file and copy the file to hdfs.

    Once the job is done you can check the value of those properties in the file stored in hdfs.

    Hope this helps.