Parallel.ForEach missing items

Question

I've following code:

HttpContext httpContext = HttpContext.Current;
RequestContext currentContext = RequestContextManager.CurrentContext;
ILifetimeScope currentSessionScope = PlatformContext.LifeTimeScope;

ConcurrentQueue<Exception> exceptions = new ConcurrentQueue<Exception>();
ConcurrentBag<ParallelCalculateObj> forEachResult = new ConcurrentBag<ParallelCalculateObj>();
ConcurrentBag<ParallelCalculateObj> testForEachPassResult = new ConcurrentBag<ParallelCalculateObj>();

ParallelLoopResult loopResult = Parallel.ForEach(applications, () =>
{
    HttpContext.Current = httpContext;
    RequestContextManager.SetCustomCurrentContext(currentContext);
    PlatformContext.LifeTimeScope = currentSessionScope;
    return new ParallelCalculateObj();
}, (application, pls, localObj) =>
{
    try
    {
        // some code
    }
    catch (Exception e)
    {
        exceptions.Enqueue(e);
    }
    testForEachPassResult.Add(localObj);
    return localObj;
}, forEachResult.Add);

where applications.Count = 3. After executing above code I got forEachResult.Count = 2 and testForEachPassResult.Count = 3

Why forEachResult collection does not contain all of elements? There is no exception and ParallelLoopResult.IsCompleted = true.

One thing that might be helpful in resolving my problem is that this three items was run under two threads:

1. Item01 -> Thread.CurrentThread.ManagedThreadId is 14
2. Item02 -> Thread.CurrentThread.ManagedThreadId is 10
3. Item03 -> Thread.CurrentThread.ManagedThreadId is 14

Answer 1

I think you are using Parallel.ForEach in a wrong way.

You are using the overload which has a local state. This local state is unique for a partition/thread, but not every iteration has a unique local state.

Think of dividing the input list into N partitions. Then there are N local states. As the final step you have combine those N local states into your final value. In general N will be less than the number of items in the list and unless you use one of the more specific overloads the TPL will determine the way the list is partitioned.

Since you apparently want to fill some list with the result of every iteration your local state should be also a list which contains the result of every iteration of that particular partition. For the final action you combine all lists into one single list:

Parallel.ForEach(
    applications,
    () => new List<ParallelCalculateObj>(),
    (application, pls, localObj) =>
    {
        // do something
        var obj = new ParallelCalculateObj { /* data of the iteration */ };
        localObj.Add(obj);
        return localObj;
    },
    localObj =>
    {
        foreach (var result in localObj)
        {
            forEachResult.Add(result);
        }
    });

Please note that if you do it like that then the order of values in forEachResult will not correspond to the order of items in applications. If you want that then you have to use the index of the ParallelLoopState class.