Search code examples
javaarrayshadoopintcloudera

hadoop passing int array from map to reducer and as output

I'm learning about using hadoop and came across the following problem: I need to pass an int array from the map function to the reducer and then as output.

So it looks something like this:

public void map(LongWritable key,Text value,OutputCollector<Text, IntWritable> output,Reporter reporter) throws IOException{
..snip..
int[] output={0,0,1,1}; //or something like it
output.collect(word,output);
}

and 

public static class reduce exteds mapReduceBase implements reducer<Text,IntWritable,Text, Intwriteable>{
  int[] sum={0,0,0,0}
  while(values.hasNext()){
   int[] numbers=values.next().get
    for(int i=0;i<numbers.length;i++)
       sum[i]=sum[i]+numbers[i]; 
  }
}

If you know how to fix this please post. thanks.

Solution

  • Look into writing your own custom Writable (it should be easy to extend ArrayWritable - in fact the JavaDoc for this class has an example for IntWritable array).