Why Parallel Foreach doesn't return always the same result in just adding int values

Question

I made two functions like below

one is just adding int values with linq

private long GetSumList(List<long> numbers)
{
    return numbers.Sum();
}

the other one is adding int values with Parallel.Foreach

private long GetSumListWithParallel(List<long> numbers)
{
    long retv = 0;

    Parallel.ForEach(numbers, number =>
    {
        retv += number; // It returns different return whenever I execute it.
        //Interlocked.Add(ref retv, number); // It returns always the same value whenever I execute it.
    });

    return retv;
}

I thought I didn't need to care about race condition, when I just adding integer numbers. Just because I thought that adding numbers had nothing to do with the order in which they were executed. if there were subtraction or multiplication, I would have care about race condition.

Why does this function return always different values? And What is the thing I should care about making parallel code?

Answer 1

The problem is not mathematical, but to do with atomicity and thread safety. That is to say, integer addition is commutative mathematically (order does not count), but += is not atomic,

i.e., if two parallel thread do a += operation at the same time, then the result is unpredictable. That's exactly why Interlocked.Add exists.