I'm trying to load data from HBase (1.1.2) using Pig (0.15.0). I've tried registering too many JARs without any success. I'm able to access it with Java but not with Pig. Below I've registered all JARs. the first five, then lib/*.jar then target/alternateLocation/*.jar which my Java code was using. Everything is installed on my local PC and HBase is running on HDFS.
--REGISTER /home/halmousli/hbase-1.1.2/lib/*.jar
--REGISTER /home/halmousli/mygit/Hbase/apachehbase/target/alternateLocation/*.jar
set hbase.zookeeper.quorum 'localhost'
data = LOAD 'hbase://emp' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('professional:designation', '-loadKey true')
as (id:bytearray,prof:CHARARRAY);
DUMP data;
pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[professional:designation, -loadKey true]' Failed to parse: Pig script failed to parse: pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[professional:designation, -loadKey true]' at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199) at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1735) at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1443) at org.apache.pig.PigServer.parseAndBuild(PigServer.java:387) at org.apache.pig.PigServer.executeBatch(PigServer.java:412) at org.apache.pig.PigServer.executeBatch(PigServer.java:398) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171) at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:749) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:376) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) at org.apache.pig.Main.run(Main.java:631) at org.apache.pig.Main.main(Main.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: /home/halmousli/mygit/Pig/mavenpigtester/src/main/resources/hbaseGetter.pig, line 21, column 7> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[professional:designation, -loadKey true]' at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:897) at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568) at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625) at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102) at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560) at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421) at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191) ... 19 more Caused by: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[professional:designation, -loadKey true]' at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:772) at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:885) ... 25 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:740) ... 26 more
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V at org.apache.pig.backend.hadoop.hbase.HBaseStorage.initScan(HBaseStorage.java:427) at org.apache.pig.backend.hadoop.hbase.HBaseStorage.(HBaseStorage.java:368) ... 31 more 2015-12-30 11:19:14,794 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: Pig script failed to parse: pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[professional:designation, -loadKey true]' Details at logfile: /home/halmousli/hadoop-2.6.0/sbin/pig_1451492353570.log
I do not know why I am seeing caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks. In my Java code, I am able to find that function and run it but not in Pig and I am using the same jars.
Scan scan = new Scan();
scan.setCacheBlocks(true);
Any suggestion to solve this problem or to other way to access HBase from Pig?. Thanks.
The reason for the error above was incompatibility between my hadoop version and hbase. I installed hbase-0.98.16-hadoop2 and the pig code is running without errors.
Generally when you see java.lang.NoSuchMethodError then it can be an indication of incompatibility between versions.