Search code examples
hiveapache-drill

Apache Drill - Hive Integration: Drill Not listing Tables

I have been trying to integrate Apache Drill with Hive using Hive Storage Plugin configuration. I configured the storage plugin with all the necessary properties required. On Drill Shell, I can view the Hive Databases using:

Show Databases;

But when i try to list tables using: 

Show Tables;

I get no results (No List of Tables).

Below are the steps i have followed from Apache Drill documentation and other sources:

  1. I created a Drill Distributed Cluster by updating drill-override.conf with same cluster id on all nodes along with ZK IP with Port and then invoking drillbit.sh on each node.
  2. Started Drill shell using drill-conf, Ensured that the Hive metastore service is active as well.

Below is configuration made in Hive Storage Plugin for Drill (from its Web-UI):

{
  "type": "hive",
  "configProps": {
    "hive.metastore.uris": "thrift://node02.cluster.com:9083",
    "javax.jdo.option.ConnectionURL": "jdbc:mysql://node02.cluster.com/hive",
    "hive.metastore.warehouse.dir": "/apps/hive/warehouse",
    "fs.default.name": "hdfs://node01.cluster.com:8020",
    "hive.metastore.sasl.enabled": "false"
  },
  "enabled": true
}

All the properties are set after referring to hive-site.xml

So, That's what all others have done to integrate Drill with Hive. Am i missing something here?

Regarding Versions- Drill: 1.14 ,Hive : 1.2 (Hive Metastore: MySQL) We also have Hive Server2 on the same nodes, is that causing any issues? I just want to integrate Drill with Hive 1.2, am i doing it right? Any pointers will be helpful, have spent nearly 2 days to get it right. Thanks for your time.

Solution

  • Starting from Drill 1.13 version Drill leverages Hive client 2.3.2 version. It is recommended to use Hive 2.3 version to avoid unpredictable issues.

    Regarding your setup, please remove all configProps except hive.metastore.uris. The other configs can be default (it is in HiveConf.java) or can be specified in your hive-site.xml.

    Also in case of empty result after usage Show Tables; even after executing use hive, check for errors in Drill's log files. If some error is there, you can create the Jira ticket to improve the output from Drill to reflect that issue.