Search code examples
teradatasqoopoozie

Sqoop failing when run from oozie but not in command line

Sqoop action is failing when run from oozie but when the same command is copy pasted in command line (edge node) it is perfectly executing fine. When run from oozie job id is triggered and then i am getting following error

Error: com.teradata.connector.common.exception.ConnectorException: index outof boundary
at com.teradata.connector.teradata.converter.TeradataConverter.convert(TeradataConverter.java:161)
at com.teradata.connector.common.ConnectorOutputFormat$ConnectorFileRecordWriter.write(ConnectorOutputFormat.java:111)
at com.teradata.connector.common.ConnectorOutputFormat$ConnectorFileRecordWriter.write(ConnectorOutputFormat.java:70)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:658)
at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at com.teradata.connector.common.ConnectorMMapper.map(ConnectorMMapper.java:134)
at com.teradata.connector.common.ConnectorMMapper.run(ConnectorMMapper.java:122)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
Intercepting System.exit(1)
Failing Oozie Launcher, Main class   [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

Following is part of workflow.xml file

<sqoop xmlns="uri:oozie:sqoop-action:0.2">
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <configuration>
            <property>
                <name>mapred.job.queue.name</name>
                <value>${queueName}</value>
            </property>
        </configuration>
    <command>export -Dhadoop.security.credential.provider.path=pathprovided   --connect connectionstring --connection-manager org.apache.sqoop.teradata.TeradataConnManager --username name --password-alias password --table name --export-dir location --input-fields-terminated-by '|'</command>
    </sqoop>

I have 2 other actions(hive,pig) in oozie executing correctly, also i have enabled parameter to use oozie sharedlib(-D oozie.use.system.libpath=$oozieusesystemlibpath). Can some one please help me

Thanks

Solution

  • The error is misleading. It is working when i removed single quotes in fields-terminated-by. So following is the code in workflow.xml

    <command>export -Dhadoop.security.credential.provider.path=pathprovided   --connect connectionstring --connection-manager org.apache.sqoop.teradata.TeradataConnManager --username name --password-alias password --table name --export-dir location --input-fields-terminated-by |</command>