Search code examples
jdbchivecloudera-quickstart-vm

Using jdbc to connect with hive in cloudera docker

I have a cloudera docker container installed in my local, also the hive port is configured, like this docker run --hostname=quickstart.cloudera --privileged=true -t -i -p 8888:8888 -p 80:80 -p 10000:10000 --name cloudera2 cloudera/quickstart /usr/bin/docker-quickstart

I want to connect it with JDBC, my code is like this,

val driver = "org.apache.hive.jdbc.HiveDriver"
val url = "jdbc:hive2://localhost:10000/default"
val username = ""
val password = ""

// there's probably a better way to do this
var connection: Connection = null

try {
  // make the connection
  Class.forName(driver)

} catch {
  case e => e.printStackTrace
}
connection = DriverManager.getConnection(url, username, password)
connection.close()

but occur an NoClassDefFoundError when I try to execute it

log4j:WARN No appenders could be found for logger (org.apache.hive.jdbc.Utils).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
    at org.apache.hive.jdbc.HiveConnection.createUnderlyingTransport(HiveConnection.java:362)
    at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:382)
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:193)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:167)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:247)
    at ScalaJdbcConnectSelect$.main(ScalaJdbcConnectSelect.scala:32)

hive version: 

Hive 1.1.0-cdh5.7.0

maven dependency

    <dependency>
        <groupId>org.apache.hive</groupId>
        <artifactId>hive-jdbc</artifactId>
        <version>1.1.0-cdh5.7.0</version>
    </dependency>

I am not sure it's because the username and password, but I've tried "cloudera","cloudera" "hive","", and "",""

Solution

  • I found it need to add hadoop common dependency, in my case as follows.

        <!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common -->
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-common</artifactId>
        <version>2.6.0-cdh5.7.0</version>
    </dependency>

    And it works fine.