Search code examples
ooziehuecloudera-cdh

How to make Hue - Oozie workflow run a java job which has config file?


I have a buildModel.jar, and a folder "conf" which contain a configuration file named config.properties.

The command line running it look like this:

hadoop jar /home/user1/buildModel.jar -t fp-purchased-products -i hdfs://Hadoop238:8020/user/user2/recommend_data/bought_together

After doing some analyze, it use the db information in "config.properties" file to store data to a mongo db.

Now i need to run it with Hue Oozie workflow, so I used Hue to upload the jar file and folder "conf" to hdfs then created a workflow. I also added "config.properties" file in workflow

This is the workflow.xml

<workflow-app name="test_service" xmlns="uri:oozie:workflow:0.4">
<start to="run_java_file"/>
<action name="run_java_file">
    <java>
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <main-class>xxx.xxx.recommender.buildModel.Application</main-class>
        <arg>-t=fp-purchased-products</arg>
        <arg>-i=hdfs://Hadoop238:8020/user/user2/recommend_data/bought_together</arg>
        <file>/user/user2/service/build_model/conf/config.properties#config.properties</file>
    </java>
    <ok to="end"/>
    <error to="kill"/>
</action>
<kill name="kill">
    <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>

And this is the workflow-metadata.json

{"attributes": {"deployment_dir": "/user/hue/oozie/workspaces/_user2_-oozie-31-1416890719.12", "description": ""}, "nodes": {"run_java_file": {"attributes": {"jar_path": "/user/user2/service/build_model/buildModel.jar"}}}, "version": "0.0.1"}

After doing analyze, it got error when save data to mongo db. It seem that the java file can't see the config.properties.

Can anyone guide me how to use Hue Oozie run java which has config file ?


Solution

  • Sorry for late answer.

    As Romain explained above. Hue will copy the config.properties to the same directory with the BuildModel.jar. So i changed the code to let BuildModel.jar read config file at the same directory. It worked !