Search code examples
swiftmathfloating-pointieee-754

Why this happens in floating point conversion?

I noticed that some floating points converted differently. This question helps me about floating points however still don't know why this happens? I added two screenshots from debug mode about the sample code. Example values : 7.37 and 9.37. Encountered it in swift and surely swift uses IEEE 754 floating point standard Pls explain how this happens? How conversion ended differently ?

if let text = textField.text {
  if let number = formatter.number(from: text) {
     return Double(number)
  }
  return nil
}

enter image description here

enter image description here

Solution

  • Double floating point numbers are stored in base-2, and cannot represent all decimals exactly.

    In this case, 7.37 and 9.37 are rounded to the nearest floating point numbers which are 7.37000000000000010658141036401502788066864013671875 and 9.3699999999999992184029906638897955417633056640625, respectively.

    Of course, such decimal representations are too unwieldy for general use, so programming languages typically print shorter approximate decimal representations. Two popular choices are

    1. The shortest string that will be correctly rounded to the original number (which in this case are 7.37 and 9.37, respectively).
    2. Rounded 17 significant digits, which is guaranteed to give the correct value when converting back to binary.

    These appear to correspond to the 2 debug output values that you are seeing.