This is the method I am calling to essentially calculate the mean of 1000 integers:
public void setMeanSampleSize(ResultData[] R) {
double temp = 0;
for(int i = 0;i < R.length;i++){
temp = temp + R[i].getTotal();
}
MeanSampleSize = (int) temp/R.length;
}
The variable Total which is part of the ResultData object is an integer.
In 99.99% of the cases, this works correctly but in some odd cases, the code yields an incorrect result of 2147483 for MeanSampleSize (which is eerily similar to 2,147,483,647 the 32-bit integer max).
Can anyone help understand why this division will work sometimes and give completely erroneous result of 2147483 on other occasions?
Thanks in advance!
You're building up your total in a double
, and then doing this:
MeanSampleSize = (int) temp/R.length;
which is
MeanSampleSize = ((int) temp)/R.length;
As Radiodef said, if temp
's value is greater than Integer.MAX_VALUE
, (int)temp
will be Integer.MAX_VALUE
(2147483647), which when divided by 1000 (with integer division) is ... 2147483.
You probably wanted to divide first (as a double), then truncate to int:
MeanSampleSize = (int) (temp / R.length);
Worth noting the various comments, though, that doing this with a double
probably isn't your best bet unless getTotal
returns a double
and you want those fractional values to accumulate. But it may be worth using a long
(and adjusting your parens as above).