Search code examples
azuresqoopoozieazure-hdinsight

How to schedule sqoop job command using oozie in azure hdinsight(remote machine)


I am trying to schedule sqoop job using oozie in hdinsight(remote machine). I have executed the sqoop job command in cmd and trying to schedule "job --exec job1" in oozie. But this is not working. I seen logs in azure blob and oozie logs there is no error. I seen success in yarn app and oozie shows failure/killed1 error.

if i am running the same command in cmd it works well.

This is my sqoop job command :

sqoop job --create job1  -- import --connect "jdbc:sqlserver://<ip:port>;database=<dbname>;username=<name>;password=<pwd>" --table tablename --target-dir /example/sqoopoutput --incremental append --check-column latestdate --last-value "1991-01-01 00:00:00.000"

I am getting this error :

[org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

i tried with all jar files in sqoop and oozie share lib jars.


Solution

  • while scheduling sqoop job in oozie you do not leave space in date value you passed, instead of space use T

    change your command as follows :

    job --create job1  -- import --connect "jdbc:sqlserver://<ip:port>;database=<dbname>;username=<name>;password=<pwd>" --table tablename --target-dir /example/sqoopoutput --incremental append --check-column latestdate --last-value "1991-01-01T00:00:00.000"