Search code examples
rprecisionfloating-accuracy

Why we get to approach asymptotically the value 0 "more" than the value 1?


Probably this is elementary for the people here. I am just a computer user.

I fooled around near the extreme values (0 and 1) for the Standard Normal cumulative distribution function (CDF), and I noticed that we can get very small probability values for large negative values of the variable, but we do not get the same reach towards the other end, for large positive values, where the value "1" appears already for much smaller (in absolute terms) values of the variable.

From a theoretical point of view, the tail probabilities of the Standard Normal distribution are symmetric around zero, so the probability mass to the left of, say, X=-10, is the same as the probability mass to the right of X=10. So at X=-10 the distance of the CDF from zero is the same as is its distance from unity at X=10. But the computer/software complex doesn't give me this.

Is there something in the way our computers and software (usually) compute, that creates this asymmetric phenomenon, while the actual relation is symmetric?

Computations where done in "r", with an ordinary laptop.

This post is related, Getting high precision values from qnorm in the tail


Solution

  • Floating-point formats represent numbers as a sign s (+1 or −1), a significand f, and an exponent e. Each format has some fixed base b, so the number represented is sfbe, and f is restricted to be in [1, b) and to be expressible as a base-b numeral of some fixed number p of digits. These formats can represent numbers very close to zero by making e very small. But the closest they can get to 1 (aside from 1 itself) is where either f is as near 1 as it can get (aside from 1 itself) and e is 0 or f is as near b as it can get and e is −1.

    For example, in the IEEE-754 binary64 format, commonly used for double in many languages and implementations, b is two, and p is 53, and e can be as low as −1022 for normal numbers (there are subnormal numbers that can be smaller). This means the smallest representable normal number is 2−1022. But near 1, either e is 0 and f is 1+2−52 or e is −1 and f is 2−2−52. The latter number is closer to 1; it is sfbe = +1•(2−2−52)•2−1 = 1−2−53.

    So, in this format, we can get to a distance of 2−1022 from zero (closer with subnormal numbers), but only to a distance of 2−53 from 1.