Search code examples
c++floating-pointdoubleieee-754

On a float rounding error


I do not understand the output of the following program:

int main()
{
    float  x     = 14.567729f;
    float  sqr   = x * x;
    float  diff1 = sqr - x * x;
    double diff2 = double(sqr) - double(x) * double(x);
    std::cout << diff1 << std::endl;
    std::cout << diff2 << std::endl;
    return 0;
}

Output:

6.63225e-006
6.63225e-006

I use VS2010, x86 compiler.

I expect to get a different output

0
6.63225e-006

Why diff1 is not equal to 0? To calculate sqr - x * x compiler increases float precision to double. Why?


Solution

  • float  diff1 = sqr - x * x;
    double diff2 = double(sqr) - double(x) * double(x);
    

    Why diff1 is not equal to 0?

    Because you have already cached sqr = x*x and forced its representation to be a float.

    To calculate sqr - x * x compiler increases float precision to double. Why?

    Because that is how C did things back before there was a C standard. I don't think modern compilers are bound to that convention, but many still do follow it. If this is the case, the right-hand sides of the calculations of diff1 and diff2 will be identical. The only difference is that after calculating the right-hand side of float diff1 = ..., the double result is converted back to a float.