Search code examples
c++floating-pointfloating-point-precision

Understanding floating point precision

Is it the case that:

  1. Representable floating point values are densest in the real number line near zero?
  2. Representable floating point values grow sparser (exponentially?) as the number line moves away from zero?
  3. If the above two are true, does that mean there is less precision farther from zero?

Overall question: Does precision in some way refer to or depend on the density of numbers you can represent (accurately)?

Solution

  • The term precision usually refers to the number of significant digits (bits) in the represented value. So precision varies with the number of bits (or digits) in the mantissa of representation. Distance from the origin has no role.

    What you say is true about the density of floats on the real line. But in this case the right term is accuracy, not precision. FP numbers of small magnitude are far more accurate that larger ones. This contrasts with integers, which have uniform accuracy over their ranges.

    I highly recommend the paper What Every Computer Scientist Should Know About Floating Point Arithmetic, which covers this and much more.