Search code examples
hadoophiveazure-hdinsighttez

Add Filesystem to Hive with Tez execution engine

I receive the following error when executing a query with the Tez execution engine against an external ORC table:

Vertex failed, vertexName=Map 6, vertexId=vertex_1504790331090_0008_1_01, diagnostics=[Vertex vertex_1504790331090_0008_1_01 [Map 6] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: item initializer failed, 
vertex=vertex_1504790331090_0008_1_01 [Map 6], java.lang.RuntimeException: 
java.lang.ClassNotFoundException: Class org.apache.
hadoop.fs.nfs.NFSv3FileSystem not found
    at 
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2241)
    at 

org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2780)
    at 
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2793)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
    at 
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2829)

However, when I run the query with MapReduce engine, it works fine. I have added the correct lines to core-site.xml, I've tried adding the jar files that act as drivers for the protocol to hive-site.xml.

Where do I need to add jars or modify configs in order for Hive+Tez to support an additional filesystem?

Solution

  • In the tez-site.xml file, configure the tez.lib.uris property to include path to required jar.