Search code examples
bigsql

Load hadoop with sql query option - failing


I am trying to use LOAD HADOOP command to copy data from teradata source.

 LOAD HADOOP USING JDBC CONNECTION URL 'jdbc:teradata://<<ip_address>>/database=EDW01_V_LV_BASE' WITH PARAMETERS ('user' = 'USR1','password'='pass1') FROM SQL QUERY 'SELECT COL1, COL2, COL3, COL4, COL5, COL6, COl7, COl8, COl9, COL10 FROM POS_RTL_STR WHERE TBL_TMST > '2015-12-15 03:49:33' AND TBL_TMST <= '2015-12-15 14:54:05' AND $CONDITIONS' SPLIT COLUMN COL1 INTO TABLE SCHEMA1.TABLE1 APPEND;.

Now load is failing with the error

2015-12-17 04:02:54,219 ERROR com.ibm.biginsights.ie.db.DBImportImpl [Thread-3] : Error during import
java.lang.RuntimeException: java.sql.SQLException: [Teradata Database] [TeraJDBC 15.00.00.20] [Error 5628] [SQLState HY000] Column 2015-12-15 03:49:33 not found in EDW01_V_LV_BASE.POS_RTL_STR.
        at com.ibm.biginsights.ie.sqoop.td.TeradataFastManager.getColumnTypesForRawQuery(TeradataFastManager.java:181)
        at org.apache.sqoop.manager.SqlManager.getColumnTypesForQuery(SqlManager.java:234)
        at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:304)
        at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
        at org.apache.sqoop.manager.BIImportTool.importTable(BIImportTool.java:97)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
        at com.ibm.biginsights.ie.db.SqoopUtils.runSqoopTool(SqoopUtils.java:146)
        at com.ibm.biginsights.ie.db.DBImportImpl.importData(DBImportImpl.java:159)
        at com.ibm.biginsights.ie.impl.ImporterImpl.executeImport(ImporterImpl.java:504)
        at com.ibm.biginsights.ie.impl.ImporterImpl.executePerformImport(ImporterImpl.java:417)
        at com.ibm.biginsights.ie.impl.ImporterImpl.performImport(ImporterImpl.java:264)
        at com.ibm.biginsights.biga.udf.LoadTool.performImport(LoadTool.java:214)
        at com.ibm.biginsights.biga.udf.BIGSQL_DDL.doLoadStatement(BIGSQL_DDL.java:644)
        at com.ibm.biginsights.biga.udf.BIGSQL_DDL.processDDL(BIGSQL_DDL.java:207)
Caused by: java.sql.SQLException: [Teradata Database] [TeraJDBC 15.00.00.20] [Error 5628] [SQLState HY000] Column 2015-12-15 03:49:33 not found in EDW01_V_LV_BASE.POS_RTL_STR.
        at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDatabaseSQLException(ErrorFactory.java:308)
        at com.teradata.jdbc.jdbc_4.statemachine.ReceiveInitSubState.action(ReceiveInitSubState.java:109)
        at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.subStateMachine(StatementReceiveState.java:307)
        at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.action(StatementReceiveState.java:196)
        at com.teradata.jdbc.jdbc_4.statemachine.StatementController.runBody(StatementController.java:123)
        at com.teradata.jdbc.jdbc_4.statemachine.StatementController.run(StatementController.java:114)
        at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:385)
        at com.teradata.jdbc.jdbc_4.TDStatement.prepareRequest(TDStatement.java:569)
        at com.teradata.jdbc.jdbc_4.TDPreparedStatement.<init>(TDPreparedStatement.java:117)
        at com.teradata.jdbc.jdk6.JDK6_SQL_PreparedStatement.<init>(JDK6_SQL_PreparedStatement.java:29)
        at com.teradata.jdbc.jdk6.JDK6_SQL_Connection.constructPreparedStatement(JDK6_SQL_Connection.java:81)
        at com.teradata.jdbc.jdbc_4.TDSession.prepareStatement(TDSession.java:1357)
        at com.teradata.jdbc.jdbc_4.TDSession.prepareStatement(TDSession.java:1401)
        at com.teradata.jdbc.jdbc.ManagerConnectionBase.prepareStatement(ManagerConnectionBase.java:302)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:752)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:775)
        at com.ibm.biginsights.ie.sqoop.td.TeradataFastManager.executeStmt(TeradataFastManager.java:191)
        at com.ibm.biginsights.ie.sqoop.td.TeradataFastManager.getColumnTypesForRawQuery(TeradataFastManager.java:177)
        ... 18 more
2015-12-17 04:02:54,219 ERROR com.ibm.biginsights.ie.db.DBImportImpl [Thread-3] : [BSL-0-1ae1ada8b]: Error during import (Job Id = ):java.sql.SQLException: [Teradata Database] [TeraJDBC 15.00.00.20] [Error 5628] [SQLState HY000] Column 2015-12-15 03:49:33 not found in EDW01_V_LV_BASE.POS_RTL_STR.

I think it is somewhere wrong in the where clause where I have used value within single quote in where clause. How should it be specified? Or there is any other error?


Solution

  • Please visit: https://www.ibm.com/support/knowledgecenter/?lang=en#!/SSPT3X_4.1.0/com.ibm.swg.im.infosphere.biginsights.db2biga.doc/doc/biga_load_from.html?cp=SSPT3X_4.1.0%2F4-4-0

    In the section FROM SQL QUERYselect-statement:

    Surround the value for QUERY with single quotes. If a single quote is needed inside the value then use two single quotes. … FROM SQL QUERY 'SELECT ID, NAME, STATE, CITY FROM CITIES WHERE CITY=''San Jose'' AND $CONDITIONS' ...