Search code examples
javamathfloating-pointdecimalfractions

Floating-Point Arithmetic = What is the worst precision/difference from Dec to Binary?


as all know decimal fractions (like 0.1) , when stored as floating point (like double or float) will be internally represented in "binary format" (IEEE 754). And some decimal fractions can not directly be represented in binary format.

What I have not understood, is the precision of this "conversion":

1.) A Floating point itself can have a precision (that is the "significant")?

2.) But also the conversion from decimal fraction to binary fraction has a precision loss?

Question:

What is the worst case precision loss (for "all" possible decimal fractions) when converting from decimal fractions to floating point fractions?

(The reason I want to know this is, when comparing decimal fractions with binary/floating point fractions I need to take the precision into account...to determine if both figures are identical. And I want this precision to be as tight/precise as possible (decimal fraction == binary fraction +/- precision)

Example (only hypothetical)

0,1 dec => 0,10000001212121212121212 (binary fraction double) => precision loss 0,00000001212121212121212
0,3 dec => 0,300000282828282 (binary fraction double) => precision loss  0,000000282828282

Solution

  • It is not entirely clear to me what you are after, but you may be interested in the following paper which discusses many of the accuracy issues involved in binary/decimal conversion, including lists of hard cases.

    Vern Paxson and William Kahan. A program for testing IEEE decimal-binary conversion. May 22, 1991 http://www.icir.org/vern/papers/testbase-report.pdf