Search code examples
javainteger-division

Division by integer in Java yields incorrect value of 2147483


This is the method I am calling to essentially calculate the mean of 1000 integers:

public void setMeanSampleSize(ResultData[] R) {

    double temp = 0;
    for(int i = 0;i < R.length;i++){

        temp = temp + R[i].getTotal();
    }
    MeanSampleSize = (int) temp/R.length;
}

The variable Total which is part of the ResultData object is an integer.

In 99.99% of the cases, this works correctly but in some odd cases, the code yields an incorrect result of 2147483 for MeanSampleSize (which is eerily similar to 2,147,483,647 the 32-bit integer max).

Can anyone help understand why this division will work sometimes and give completely erroneous result of 2147483 on other occasions?

Thanks in advance!


Solution

  • You're building up your total in a double, and then doing this:

    MeanSampleSize = (int) temp/R.length;
    

    which is

    MeanSampleSize = ((int) temp)/R.length;
    

    As Radiodef said, if temp's value is greater than Integer.MAX_VALUE, (int)temp will be Integer.MAX_VALUE (2147483647), which when divided by 1000 (with integer division) is ... 2147483.

    You probably wanted to divide first (as a double), then truncate to int:

    MeanSampleSize = (int) (temp / R.length);
    

    Worth noting the various comments, though, that doing this with a double probably isn't your best bet unless getTotal returns a double and you want those fractional values to accumulate. But it may be worth using a long (and adjusting your parens as above).