Search code examples
hadoopmapreducehbase

Example for running mapreduce on hdfs files and storing reducer results in hbase table


Can somebody give one good example link for mapreduce with Hbase? My requirement is run mapreduce on hdfs file and store reducer output to hbase table. Mapper input will be hdfs file and output will be Text,IntWritable key value pairs. Reducers output will be Put object ie add reducer Iterable IntWritable values and store in hbase table.


Solution

  • Here is the code which will solve your problem



    Driver

    HBaseConfiguration conf =  HBaseConfiguration.create();
    Job job = new Job(conf,"JOB_NAME");
        job.setJarByClass(yourclass.class);
        job.setMapperClass(yourMapper.class);
        job.setMapOutputKeyClass(Text.class);
        job.setMapOutputValueClass(Intwritable.class);
        FileInputFormat.setInputPaths(job, new Path(inputPath));
        TableMapReduceUtil.initTableReducerJob(TABLE,
                yourReducer.class, job);
        job.setReducerClass(yourReducer.class);
                job.waitForCompletion(true);
    


    Mapper&Reducer

    class yourMapper extends Mapper<LongWritable, Text, Text,IntWritable> {
    //@overide map()
     }
    

    class yourReducer
            extends
            TableReducer<Text, IntWritable, 
            ImmutableBytesWritable>
    {
    //@override rdeuce()
    }