Search code examples
sqloracle-databasehivesqoop

sqoop java.lang.RuntimeException: Can't parse input data

I'm trying to export a table from hive to oracle database.

The table in hive I try to export from 

CREATE TABLE `db.to_oracle`(
  `ctn` string,
  `platform_code` string,
  `sum` double,
  `days_ctn` int,
  `mtr_comments` string,
  `imsi` string,
  `ban` int,
  `price_plan` string,
  `market_code` string,
  `currency` string,
  `balance` string,
  `time_key` date)
ROW FORMAT SERDE
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'

My sqoop statement

sqoop export \
-D mapreduce.job.queuename=adhoc \
-D oracle.jdbc.mapDateToTimestamp=false \
--connect jdbc:oracle:thin:secret \
--password pas \
--username user\
--export-dir /apps/hive/warehouse/db.db/to_oracle/ \
--input-fields-terminated-by '\t' \
--input-lines-terminated-by '\n' \
--table database.oracle_table

I get this error

java.lang.RuntimeException: Can't parse input data: '9696278394S10.01dolg_neaktivnost#20170513_3#1#12.0525099318958479562895398812VSE1NIKORUR-9992017-05-13'

It seems sqoop doesn't recognize the delimiter or something. Why is that and how do I correct it?

Solution

  • The problem is with the Delimiters, as you did not specify the delimiters in hive table, the default hive delimiters are:

    FIELDS TERMINATED BY '\001'

    COLLECTION ITEMS TERMINATED BY '\002'

    MAP KEYS TERMINATED BY '\003'

    LINES TERMINATED BY '\n'

    Change the delimiters either in hive table creation or in your sqoop job to resolve the issue.