Search code examples
hivesqoop

Is there a way to exclude the "partitioned by" column from the hive?


I created the table by entering the command below.

create external table test (A STRING, C STRING, D FLOAT) partitioned by (B STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' stored as textfile;

Then, the following commands were executed to load the data in HDFS.

load data inpath '/sqoop_test/part-m-00000' overwrite into table test partition(B='2020-03-01')

The part-m-0000 file is configured as below.

1,2020-03-01,test1,3
1,2020-03-01,test2,6
1,2020-03-01,test3,4
...

Lastly, I checked with select, and I checked that the data type was pushed back one by one due to the partition column. Like below:

1    2020-03-01    NULL    3    2020-03-01
1    2020-03-01    NULL    6    2020-03-01
1    2020-03-01    NULL    4    2020-03-01
...

How can I solve the problem of being pushed one by one and the value of the partion column at the end?


Solution

  • When importing using sqoop, exclude the column to be set as a partition column.😅😅