Search code examples
javafloating-pointgeospatialtrigonometrydouble-precision

Java: Trigonometry and double inaccuracy causing NaN

I have a distance formula using latitude and longitude:

distance = EARTH_MILES_RADIUS
    * Math.acos(Math.sin(lat1 / RADIAN_CONV)
    * Math.sin(lat2 / RADIAN_CONV)
    + Math.cos(lat1 / RADIAN_CONV)
    * Math.cos(lat2 / RADIAN_CONV)
    * Math.cos((lng2 - lng1) / RADIAN_CONV));

lat1,lng1,lat2,lng2 are double primitives. They come to me as double primitives and there is nothing I can do about it.

The problem is that when I have a pair of longitude or latitudes that are the same the formula sometimes returns NaN. I believe this is because I am taking the arc cosine of a number very slightly greater than 1, when in fact it should be exactly 1. I would probably have problems if the points were antipodal as well, where they might be slightly less than -1.

How can I best fix this problem?

Solution

  • Simply checking for equality was enough to fix my problem:

    if (lat1 == lat2 && lng1 == lng2) ...