Search code examples
javadoublelong-integerrounding-error

Checking if the result of a calculation is an integer or a decimal


I have this formula

Math.sqrt( 2 * Math.sqrt(m) + 1 / 4 ) - 1 / 2

in a function where m is the input of type long.
I need to check if the result of this expression is a whole number or a decimal, return the result if it's a whole number or return -1 if it is decimal.
So, I decided to place the result in a double:

double n = Math.sqrt( 2 * Math.sqrt(m) + (double)1 / 4 ) - (double)1 / 2;

And I tried some solutions I found here like:

  • Using modulo operator: if ( n % 1 == 0 )

  • Or using the Math.ceil and floor functions

But none of them work when the fractional part of the result of the expression is so close to 1, something like .98999...
For example when m = 2084574945554272657, the result should be 53735.96813...
but it is rounded and actually what is stored in n is 53736.0 and the answer of the function is that n is an integer, which is not correct.
Is there a way I can check the result correctly?


Solution

  • If you need precision, I'd suggest you use BigDecimal. It is more demanding on the CPU side, so you need to choose between performance and precision.

        //Just an example, apply your formula here
        BigDecimal big = BigDecimal.valueOf(6.00);
        big = big.sqrt(MathContext.DECIMAL128);
    
        if (big.compareTo(big.setScale(0, RoundingMode.DOWN)) == 0) {
            return big.longValue();
        } else {
            return -1;
        }
    

    For some reason .equals() doesn't do the trick, but with .compareTo() it seems to work fine.