I stumbled upon a weird edge behaviour of python 3's "flooring division" using //
. After some calculations I arrive at the formula (a1 - a2) // b
. Using
a1 = 226.72560000000001
a2 = 0.2856000000000165
b = 1.02
# all types are np.float64
I get a different result than had I used np.floor()
:
>>> (a1 - a2) // b
221.0
>>> np.floor((a1 - a2) / b)
222.0
This difference obviously stems from floating point calculation, but why do they behave differently? Using a higher precision calculator such as ke!san I can verify that //
's behaviour is mathematically correct, while for my calculation np.floor()
would deliver the correct result.
If x is very close to an exact integer multiple of y, it’s possible for x//y to be one larger than (x-x%y)//y due to rounding. In such cases, Python returns the latter result, in order to preserve that divmod(x,y)[0] * y + x % y be very close to x
link to doc:
https://docs.python.org/3/reference/expressions.html#id10
What should the output of:
3.3//1.1
3.0 right?
wrong it is 2.0
So, when the numerator is very close to the multiple of denominator
x//y
will return (x-x%y)//y
:
a1 = 226.72560000000001
a2 = 0.2856000000000165
b = 1.02
x = a1-a2
y = 1.02
(x-x%y)//y
#output
221.0