I am unable to import data from Mysql to Hdfs.My bashrc & sqoop-env.sh files are fine. Also I am able to run sqoop list-databases command successfully. The problem is with import command it is throwin an outputconnectionfailed exception please refer to below error & help me out:
Blockquote rahul@ubuntu:~$ sqoop import --connect jdbc:mysql://localhost/rahul --username root --password 123 --table emp -m1 --target-dir /sqoopimport/emp Warning: /usr/lib/hbase does not exist! HBase imports will fail. Please set $HBASE_HOME to the root of your HBase installation. 14/09/09 01:22:45 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 14/09/09 01:22:45 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 14/09/09 01:22:45 INFO tool.CodeGenTool: Beginning code generation 14/09/09 01:22:45 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM
empAS t LIMIT 1 14/09/09 01:22:45 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROMempAS t LIMIT 1 14/09/09 01:22:45 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/hadoop Note: /tmp/sqoop-rahul/compile/a81597835880664d34a2ff0e4c7b9b33/emp.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 14/09/09 01:22:46 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-rahul/compile/a81597835880664d34a2ff0e4c7b9b33/emp.jar 14/09/09 01:22:46 WARN manager.MySQLManager: It looks like you are importing from mysql. 14/09/09 01:22:46 WARN manager.MySQLManager: This transfer can be faster! Use the --direct 14/09/09 01:22:46 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path. 14/09/09 01:22:46 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql) 14/09/09 01:22:46 INFO mapreduce.ImportJobBase: Beginning import of emp 14/09/09 01:22:47 INFO mapred.JobClient: Running job: job_201409090100_0003 14/09/09 01:22:48 INFO mapred.JobClient: map 0% reduce 0% 14/09/09 01:22:54 INFO mapred.JobClient: Task Id : attempt_201409090100_0003_m_000000_0, Status : FAILED java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failureThe last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:193) at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:162) ... 9 more Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:355) at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2479) at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2516) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2301) at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:834) at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:47) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:416) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:317) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:215) at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:278) at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:187) ... 10 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at java.net.Socket.(Socket.java:425) at java.net.Socket.(Socket.java:241) at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:259) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:305) ... 26 more
14/09/09 01:22:54 WARN mapred.JobClient: Error reading task outputConnection refused 14/09/09 01:22:54 WARN mapred.JobClient: Error reading task outputConnection refused 14/09/09 01:22:59 INFO mapred.JobClient: Task Id : attempt_201409090100_0003_m_000000_1, Status : FAILED java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:193) at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:162) ... 9 more Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:355) at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2479) at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2516) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2301) at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:834) at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:47) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:416) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:317) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:215) at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:278) at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:187) ... 10 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at java.net.Socket.(Socket.java:425) at java.net.Socket.(Socket.java:241) at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:259) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:305) ... 26 more
14/09/09 01:22:59 WARN mapred.JobClient: Error reading task outputConnection refused 14/09/09 01:22:59 WARN mapred.JobClient: Error reading task outputConnection refused 14/09/09 01:23:03 INFO mapred.JobClient: Task Id : attempt_201409090100_0003_m_000000_2, Status : FAILED java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications l
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:193) at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:162) ... 9 more Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:355) at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2479) at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2516) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2301) at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:834) at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:47) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:416) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:317) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:215) at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:278) at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:187) ... 10 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at java.net.Socket.(Socket.java:425) at java.net.Socket.(Socket.java:241) at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:259) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:305) ... 26 more
14/09/09 01:23:03 WARN mapred.JobClient: Error reading task outputConnection refused 14/09/09 01:23:03 WARN mapred.JobClient: Error reading task outputConnection refused 14/09/09 01:23:09 INFO mapred.JobClient: Job complete: job_201409090100_0003 14/09/09 01:23:09 INFO mapred.JobClient: Counters: 6 14/09/09 01:23:09 INFO mapred.JobClient: Job Counters 14/09/09 01:23:09 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=20325 14/09/09 01:23:09 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)= 14/09/09 01:23:09 INFO mapred.JobClient:
Total time spent by all maps waiting after reserving slots (ms)=0 14/09/09 01:23:09 INFO mapred.JobClient: Launched map tasks=4 14/09/09 01:23:09 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0 14/09/09 01:23:09 INFO mapred.JobClient: Failed map tasks=1 14/09/09 01:23:09 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 23.174 seconds (0 bytes/sec) 14/09/09 01:23:09 INFO mapreduce.ImportJobBase: Retrieved 0 records. 14/09/09 01:23:09 ERROR tool.ImportTool: Error during import: Import job failed! Blockquote
I fixed it. The problem was that I was using localhost in the import statement, as the sql was running in the same system. when I used the actual IP instead of localhost. then it worked like a charm.
Also i was using root username & password to connect to sql. for some reason it didn't work.So I created another user & granted all privileges to that user.
GRANT ALL PRIVILEGES ON employee.* to 'sqoopuser'@'%' IDENTIFIED BY 'passphrase';
Mistake:sqoop import --connect jdbc:mysql://localhost/rahul --username root --password 123 --table emp -m1 --target-dir /sqoopimport/emp
Correction:sqoop import --connect jdbc:mysql://192.168.202.139:3306/rahul --username sqoopuser --password 123 --table emp -m1 --target-dir /sqoopimport/emp