OpenMP for loop cause wrong result

Question

I want to compare the loop performance with openmp by part of simple code. But the result is wrong.

I already use reduction to avoid race condition but never work.

here is my code: thanks for any suggestion

void TestMP_1(){
    float afValueTmp[MP_TEST_NUM] = { 0 }; // MP_TEST_NUM = 10000
    float sum = 0, sumNoMP = 0;
    float fDiff = 0;
    double eTDiff = 0;
    double t0 = 0;
    double t1 = 0;

    for (int i = 0; i < MP_TEST_NUM; i++)
    {
        afValueTmp[i] = i;
    }

    t0 = (double)getTickCount();
    for (int i = 0; i < MP_TEST_NUM; i++)
    {
        for (int k = 0; k < MP_TEST_NUM; k++);  // just for delay

        sumNoMP += afValueTmp[i];   // equation 4
    }

    t0 = ((double)getTickCount() - t0) / getTickFrequency();
    t1 = (double)getTickCount();

    #pragma omp parallel for reduction( +:sum)
    for (int i = 0; i < MP_TEST_NUM; i++)
    {
        for (int k = 0; k < MP_TEST_NUM; k++);  // just for delay

        sum += afValueTmp[i];
    }

    t1 = ((double)getTickCount() - t1) / getTickFrequency();
    eTDiff = t0 - t1;   // time improve
    fDiff = sum - sumNoMP;  // check result
    printf("%.3f\n", eTDiff);
}

Answer 1

You're facing floating point accuracy issues. Please allow me to elaborate:

#include <stdio.h>

int main(void)
{
    float myOrigNumber = 49995000;
    float myNumber = myOrigNumber + 1.;

    printf ("orig: %f new: %f diff: %f\n",
            myOrigNumber, myNumber, myNumber-myOrigNumber);
    return 0;
}

The result will be:

orig: 49995000.000000 new: 49995000.000000 diff: 0.000000

So, where did that +1 go?

The float type only has 7 to 8 significant digits. It doesn't matter where they are, because floats are internally always represented in Scientific notation as x.xxE+yy notation, where x.xx has 24 bits and yy has 8 bits. The number 49995001 is larger than 2^24 (16,777,216), so it will be rounded to the closest number that can be accurately represented, which apparently is 49995000.

This is why using double for sum will alleviate your pain. It's not a real solution, though. Reduction operations have the requirement that the operation must be commutative. But that's not necessarily the case for floating point addition: if you add hundred times 1 and then 49995000 to sum, the result will be different from when you first add 1 and 49995000, and then ninety-nine times 1: in the second case, every later +1 will be rounded down as shown above.