Search code examples
mysqlhadoophivesqoop

Using Sqoop to import data from MySQL to Hive

I am using Sqoop (version 1.4.4) to import data from MySQL to Hive. The data will be a subset of one of tables, i.e. few columns from a table. Is it necessary to create table in Hive before hand. Or importing the data will create the name specified in the command if it is not in the Hive?

Solution

  • As mentioned in the sqoop documentation, you will not have to create any hive tables if you use the --hive-import argument in your command

    example:

    sqoop import \
--connect jdbc:mysql://mysql_server:3306/db_name \
--username mysql_user \
--password mysql_pass \
--table table_name \
--hive-import

    Also... consider the --hive-overwrite argument if you want to schedule a full data import, on a daily base for example