Search code examples
mapreducehbaseoozie

How to schedule Hbase Map-Reduce Job by oozie?


I want to schedule a Hbase Map-Reduce job by Oozie.I am facing following problem .

How/Where  to specify these properties in oozie workflow ?
(    i> Table name for Mapper/Reducer 
     ii> scan object for Mapper         )


    Scan scan = new Scan(new Get());

    scan.setMaxVersions();

    scan.addColumn(Bytes.toBytes(FAMILY),
            Bytes.toBytes(VALUE));
    scan.addColumn(Bytes.toBytes(FAMILY),
            Bytes.toBytes(DATE));

    Job job = new Job(conf, JOB_NAME + "_" + TABLE_USER);
    // These two properties :-
    TableMapReduceUtil.initTableMapperJob(TABLE_USER, scan,
            Mapper.class, Text.class, Text.class, job);
    TableMapReduceUtil.initTableReducerJob(DETAILS_TABLE,
            Reducer.class, job);

or

please let me know the best way to schedule a Hbase Map-Reduce Job by Oozie .

Thanks :) :)


Solution

  • The best way(According to me ) to schedule a Hbase Map_Reduce job is to schedule it as a .java file . It works well and there is no need to write code to change your scan to string , etc. So i am scheduling my jobs like java file till i get any better option .

    workflow-app xmlns="uri:oozie:workflow:0.1" name="java-main-wf">
    <start to="java-node"/>
    <action name="java-node">
        <java>
              <job-tracker></job-tracker>
            <name-node></name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <main-class>org.apache.oozie.example.DemoJavaMain</main-class>
            <arg>Hello</arg>
            <arg>Oozie!</arg>
         <arg>This</arg>
            <arg>is</arg>
         <arg>Demo</arg>
            <arg>Oozie!</arg>
    
        </java>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>Java failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>