How to correctly add or subtract using floats? For example how to perform:
2.4e-07 - 1e-8
so that it returns 2.3e-7
instead of 2.2999999999999997e-07
.
Converting to int first yields unexpected results, the below returns 2.2e-07
:
int(2.4e-07 * 1e8 - 1) * 1e-8
Similarly,
(2.4e-07 * 1e8 - 1) * 1e-8
returns 2.2999999999999997e-07
.
How to perform subtraction and addition of numbers with 8 decimal point precision?
2.2999999999999997e-07
is not sufficient as the number is used as a lookup in a dictionary, and the key is 2.3e-7
. This means that any value other than 2.3e-7
results in an incorrect lookup.
I suggest using the decimal
data type (it is present in the stardard installation of Python), because it uses fixed precision to avoid just the differences you are talking about.
>>> from decimal import Decimal
>>> x = Decimal('2.4e-7')
>>> x
Decimal('2.4E-7')
>>> y = Decimal('1e-8')
>>> y
Decimal('1E-8')
>>> x - y
Decimal('2.3E-7')