Search code examples
architecturefloating-pointprecision32-bitfloating-accuracy

Understanding 32 bit floats

I'm trying to better understand floating points in computers. During my reading about 32 bit floats, I found that generally speaking, 32 point floats are constructed as followed:

  • 1 bit for positive/negative.
  • 23 bits are reserved for the mantissa. This means we can have 2^23 unique mantissas, with 7 significant digits.
  • 8 bits are reserved for the exponent. What I read says that "half of the 2^8 numbers can be used for positive exponents, while 2^8 can be used for the negative". I also read that the exponent can vary from 10^-38 -> 10^38.

I have few questions about this..

  • I don't understand why half the exponent bits are positive, and half are negative. Couldn't there be more precision if the first of the 8 bits was used to indicate positive or negative, like the first bit in the floating point architecture. Then there could be 2^7 positive exponents, and 2^7 negative exponents?
  • With 2^7, where does 10^38 come from? Those are vastly different.
  • If the mantissa is 2^23, why is the significant digits limited to 7 significant digits? The book indicates that it's because 2^23 is similar to 10^7, but they seem to be pretty far apart, and I'm not sure why that would be the limit?
Solution

  • 1 bit for positive/negative.

    Yes.

    23 bits are reserved for the mantissa. This means we can have 2^23 unique mantissas, with 7 significant digits.

    23 bits are used for the field that contains the primary encoding of the significand. Another bit is encoded via the exponent field. The full mathematical significand has 24 bits. (“Significand” is the preferred term for the fraction portion of a floating-point representation. “Mantissa” is an old term for the fraction portion of a logarithm. Mantissas are logarithmic; adding to a mantissa multiplies the number represented. Significands are linear; adding to a significand adds to the number represented.)

    8 bits are reserved for the exponent.

    Yes.

    What I read says that "half of the 2^8 numbers can be used for positive exponents, while 2^8 can be used for the negative".

    That is incorrect. Stop using that source of information.

    I also read that the exponent can vary from 10^-38 -> 10^38.

    No, the range of finite numbers representable in this format is from about −3.4•1038 to +3.4•1038, and the smallest positive normal number represented is about 1.7•10−38. These are simply values that happen to be obtained via the format. They are not explicitly encoded into it.

    I don't understand why half the exponent bits are positive, and half our negative.

    The exponent field encodes a value in the following way. The eight bits of the exponent field are interpreted as an unsigned binary integer, E. Then:

    • If E is 255, the floating-point datum represents infinity or a NaN. (Details below).
    • If E is 0, the floating-point datum represents zero or subnormal number.
    • Otherwise, the bias 127 is subtracted. The field value E represents an exponent e = E−127.

    Couldn't there be more precision if the first of the 8 bits was used to indicate positive or negative, like the first bit in the floating point architecture. Then there could be 2^7 positive exponents, and 2^7 negative exponents?

    This question arises from the false premise about the exponents. Except for 255, all values of the exponent field are used to represent specific numbers. 255 does give up the opportunity to represent more numbers in exchange for including some information with NaNs (see below).

    With 2^7, where does 10^38 come from? Those are vastly different.

    Since 255 is used for infinities and NaNs, largest E used to represent a finite number is 254. Its corresponding e is 254−127 = 127. This is used as the exponent for two. 2127 is about 1.7•1038. Additionally, the significand may represent a value near two, so the combined value represented can be as large as about 3.4•1038. “10^38” arises from lax attention to details.

    If the mantissa is 2^23, why is the significant digits limited to 7 significant digits? The book indicates that it's because 2^23 is similar to 10^7, but they seem to be pretty far apart, and I'm not sure why that would be the limit?

    Binary floating-point formats do not contain any decimal digits. Some people “measure” their precision in terms of “about” how many decimal digits they preserve. Recall from above that the actual significand of the IEEE-754 binary32 format is 24 bits. 224 ≈ 107.22, so some people say that the 24-bit significands represent “about” 7.22 decimal digits. However, not all seven-digit decimal numbers can to converted to binary32 and then back to decimal without losing at least one digit, so this is a bad measurement. It misleads people, so it should not be used.

    To decode an IEEE-754 binary32 datum, let S be the sign bit, E be the 8 exponent bits as an unsigned integer, and F be the 23 primary significand bits, as an unsigned integer. Let s be (−1)S, so it is +1 if S is 0 and −1 if S is 1.

    • If E is 255 and F is 0, the datum represents +∞ or −∞, s•∞.
    • If E is 255 and F is not 0, the datum represents a NaN (Not a Number). In many implementations, if the high bit of F is 0, it is a signaling NaN (which generates an exception when used in arithmetic, and is a quiet NaN otherwise. The remaining bits of F may be used for other purposes, such as recording information about where the NaN originated.
    • If 0 < E < 255, the datum represents s•2E−127•(1+F•2−23). These are the normal numbers.
    • If E is 0, the datum represents s•21−127•(0+F•2−23). These are the subnormal numbers (and zero). Note two changes: E in exponent of 2 is replaced with 1, and the leading bit for the fraction portion changes from 1 to 0.