Search code examples
javahadoopmapreducehadoop2

Unable to set RCFileInputFormat as InputFormatClass in new mapreduce API

I'm trying to read RCFiles in mapper phase and I was able to achieve the same comfortably in old mapred API.

Now, I refactoring my code to use new mapreduce API.

Using Job instead of JobConf to configure job properties. But I'm unable to set RCFileInputFormat as InputFormatClass.

Below is the compilation error that i'm getting : 

job.setInputFormatClass(RCFileInputFormat.class);

The method setInputFormatClass(Class<? extends InputFormat>) in the type Job is not applicable for the arguments (Class<RCFileInputFormat>)

How can i overcome this issue and use new mapreduce API's to achieve the same functionality.

Solution

  • RCFileInputFormat uses the old MR API called mapred. You need to use one which uses the mapreduce API. Looking around you might be able to use RCFileMapReduceInputFormat from here

    It seems to have the same Key/Value signature as the one you were trying to use:

    <K extends LongWritable, V extends BytesRefArrayWritable>