I am trying to copy a file from HDFS one directory to other directory in HDFS, with the help of shell script as a part of oozie Job, but i am not able to copy it through oozie.
Can we copy file from HDFS one directory to other director in HDFS using oozie.
when i am running the oozie job, i am not any getting error.
it is showing status SUCCEEDED but file is not copying to destination directory.
oozie Files are below.
test.sh
#!/bin/bash
echo "listing files in the current directory, $PWD"
sudo hadoop fs -cp /user/cloudera/RAVIOOZIE/input/* /user/cloudera/RAVIOOZIE/output/
ls # list files
my workflow.xml is
<workflow-app name="RAMA" xmlns="uri:oozie:workflow:0.5">
<start to="shell-381c"/>
<kill name="Kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<action name="shell-381c">
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<exec>/user/cloudera/test.sh</exec>
<file>/user/cloudera/test.sh#test.sh</file>
<capture-output/>
</shell>
<ok to="End"/>
<error to="Kill"/>
</action>
<end name="End"/>
and my job.properties
oozie.use.system.libpath=True
security_enabled=False
dryrun=True
jobTracker=localhost:8032
nameNode=hdfs://quickstart.cloudera:8020
oozie.wf.application.path=${nameNode}/user/cloudera/test/
please help on this. why file is not copying to my destination director.
please let me know is there any thing i missed.
As mentioned in the comments by @Samson:
If you want to do hadoop actions with oozie, you should use a hdfs action rather than a shell action for that.
I am not sure why you don't get an error, but here is some speculation on what might happen:
You give oozie the task of starting a shell action, it succesfully starts the shell action and reports a success. Then the shell action fails, but that's not oozies problem.