Search code examples
javamysqlhadoopsqoop

How to auto-generate Java files?


I am working with importing of files between MySQL and HDFS using Sqoop. When I am working, two java files were auto-generated in my home directory. When I opened the files, it contained code that does same work as Sqoop command does. Following is my sqoop code,

sqoop import --connect "jdbc:mysql://quickstart.cloudera:3306/database" --username root --password cloudera  --query 'select * from table1 WHERE $CONDITIONS' --where "col1 != NULL OR col2 != NULL OR col3 != NULL OR col4 != NULL" --direct --target-dir '/user/cloudera/input/table1' --fields-terminated-by '|' --lines-terminated-by '\n' -m 1

I don't know how it got generated. I have rerun the code, after deleting the java files, and this time they were not generated. Can anyone tell me how it got generated, and how to generate equivalent java code file, and what this process is all about? Thanks in advance.


Solution

  • Why they are generated?
    Sqoop invokes map reduce program which is written in Java. Default it's path is in your home directory.
    You can change it's path
    By passing an argument to Sqoop command. ie. : --outdir

    Java class file will generate to import the data. Class file will get generated if you delete it and it is names after tablename.

    You can refer this link

    1. https://sqoop.apache.org/docs/1.4.1-incubating/SqoopUserGuide.html
    2. https://community.hortonworks.com/questions/25931/sqoop-import-command-generate-java-files-automatic.html