Search code examples
pythonpython-3.xfloating-pointprecisionfloating-accuracy

How to correctly deal with floating point arithmetic in Python?


How to correctly add or subtract using floats? For example how to perform:

2.4e-07 - 1e-8

so that it returns 2.3e-7 instead of 2.2999999999999997e-07.

Converting to int first yields unexpected results, the below returns 2.2e-07:

int(2.4e-07 * 1e8 - 1) * 1e-8

Similarly,

(2.4e-07 * 1e8 - 1) * 1e-8

returns 2.2999999999999997e-07.

How to perform subtraction and addition of numbers with 8 decimal point precision?

2.2999999999999997e-07 is not sufficient as the number is used as a lookup in a dictionary, and the key is 2.3e-7. This means that any value other than 2.3e-7 results in an incorrect lookup.


Solution

  • I suggest using the decimal data type (it is present in the stardard installation of Python), because it uses fixed precision to avoid just the differences you are talking about.

    >>> from decimal import Decimal
    >>> x = Decimal('2.4e-7')
    >>> x
    Decimal('2.4E-7')
    >>> y = Decimal('1e-8')
    >>> y
    Decimal('1E-8')
    >>> x - y
    Decimal('2.3E-7')