Search code examples
computer-scienceieee-754

IEEE 754 calculation discrepancy

I'm getting some discrepancies when I try to convert a float to IEEE 754 format on paper. Specifically, there is a very tiny difference between my calculated normalized mantissa and the one provided by a calculator. I end up getting the following:

0 10000101 01001100100010100011110 : 83.1349945068359375

According to this website, my least significant bit should be 1, not a 0. That's the only difference.

0 10000101 01001100100010100011111 : 83.13500213623046875

Upon closer inspection, the one provided by the calculator is slightly more accurate.

  83.13500000000000000
- 83.1349945068359375
__________________________
  00.0000054931650625

  83.13500213623046875
- 83.13500000000000000
__________________________
  00.00000213623046875

I'm pretty sure I followed the on-paper algorithm to the letter, and I think my math is tight. I suspect there some added algorithmic layer that provides this final step of accuracy. Does anyone know what it might be?

My calculations are below.

My IEEE 754 conversion of 83.135

Solution

  • You stopped when you reached the 24th bit. But the number was not completely converted. There is a remainder. If the remainder is less than ½, you should round down. If it is more, you should round up. If it is exactly ½, round to make the last bit even.