Hadoop Reducer Custom Writable

Question

I have this following Reducer class

public class CompanyMinMaxReducer extends Reducer<Text, DateClosePair, Text, Text> {
   private Text rText = new Text();

public void reduce(Text key, Iterable<DateClosePair> values, Context context)
          throws IOException, InterruptedException {

int min = Integer.MAX_VALUE;
int max = Integer.MIN_VALUE;
    LongWritable minDay = new LongWritable();
    LongWritable maxDay = new LongWritable();

for(DateClosePair val: values){
  LongWritable tempDate = val.getDate();
      DoubleWritable tempClose = val.getClose();

      if(tempDate.compareTo(maxDay) > 0){
        maxDay = tempDate;
      }else if(tempDate.compareTo(minDay) < 0){
        minDay = tempDate;
      }


      if(tempClose.get() > max){
        max = (int)tempClose.get();
      }else if(tempClose.get() < min){
        min = (int)tempClose.get();
      }
    }

String minDayFinal = "" + new SimpleDateFormat("yyyy").format(new Date(minDay.get()));
String maxDayFinal = "" + new SimpleDateFormat("yyyy").format(new Date(maxDay.get()));
    String output = minDayFinal + " - " + maxDayFinal + " MIN: " + min + " MAX: " + max;

    rText.set(output);
    context.write(key, rText);
}
}

My dataset is in the following format:

exchange, stock_symbol, date, stock_price_open,stock_price_high,stock_price_low, stock_price_close, stock_volume,stock_price_adj_close.

For example:

NASDAQ,AAPL,1970-10-22, ... 

I am asked to write a new MapReduce program that for each company provides the range of years it has been present in the stock market, and the maximum and minimum closing value obtained by the stock.

My program produces the correct output but the start date is constant for some reason:

AAON    1970 - 2002 MIN: 1 MAX: 35
AATI    1970 - 2010 MIN: 2 MAX: 15
ABCO    1970 - 2004 MIN: 14 MAX: 69
ABCW    1970 - 2007 MIN: 0 MAX: 53
ABII    1970 - 2008 MIN: 25 MAX: 78
ABIO    1970 - 1999 MIN: 0 MAX: 139
ABMC    1970 - 2004 MIN: 0 MAX: 6
ABTL    1970 - 2004 MIN: 0 MAX: 58
ACAD    1970 - 2009 MIN: 0 MAX: 17
ACAP    1970 - 2005 MIN: 15 MAX: 55
ACAT    1970 - 2009 MIN: 3 MAX: 29
ACCL    1970 - 1997 MIN: 3 MAX: 104
ACEL    1970 - 1998 MIN: 0 MAX: 10
ACET    1970 - 2004 MIN: 4 MAX: 27
ACFC    1970 - 2008 MIN: 1 MAX: 20
ACGL    1970 - 1997 MIN: 11 MAX: 80
ACLI    1970 - 2006 MIN: 2 MAX: 77
ACLS    1970 - 2001 MIN: 0 MAX: 30

The DateClosePair is a customer Writable I wrote like every example you would find on the web.

It is very odd that the min_closing price and the max_closing price are correct but the mix_date and max_date wrong.

Any thoughts?

Answer 1

I have resolved the problem which comes to be caused by aliasing.

Instead of doing maxDay = tempDate; where now maxDay is pointing to the tempDate object, I should call the method .set().

Solution:

maxDay.set(tempDate.get());