Search code examples
pythonmathsqrt

Why is math.sqrt() incorrect for large numbers?

Why does the math module return the wrong result?

First test

A = 12345678917
print 'A =',A
B = sqrt(A**2)
print 'B =',int(B)

Result

A = 12345678917
B = 12345678917

Here, the result is correct.

Second test

A = 123456758365483459347856
print 'A =',A
B = sqrt(A**2)
print 'B =',int(B)

Result

A = 123456758365483459347856
B = 123456758365483467538432

Here the result is incorrect.

Why is that the case?

Solution

  • Because math.sqrt(..) first casts the number to a floating point and floating points have a limited mantissa: it can only represent part of the number correctly. So float(A**2) is not equal to A**2. Next it calculates the math.sqrt which is also approximately correct.

    Most functions working with floating points will never be fully correct to their integer counterparts. Floating point calculations are almost inherently approximative.

    If one calculates A**2 one gets:

    >>> 12345678917**2
152415787921658292889L

    Now if one converts it to a float(..), one gets:

    >>> float(12345678917**2)
1.5241578792165828e+20

    But if you now ask whether the two are equal:

    >>> float(12345678917**2) == 12345678917**2
False

    So information has been lost while converting it to a float.

    You can read more about how floats work and why these are approximative in the Wikipedia article about IEEE-754, the formal definition on how floating points work.