Search code examples
c#.netlinqparallel-extensions

ParallelQuery with methods that take in an IEnumerable


I have a method that takes in an IEnumerable (AMethod defined below), invokes the Select extension method and returns the results. From the results returned I invoke a long running operation.

How does the behaviour of the above scenario change if the object I pass in to AMethod is a ParallelQuery?

My goal is to make this whole operation as parallel as possible. My concern is that extension methods are compile time and the 'Select' method inside of AMethod is not the parallel version even though the object I passed in is a ParalellQuery.

I assume this is a common scenario and there is a cookbook solution? Do I have to create 2 versions of every method - one that takes in/returns an IEnumerable and one that takes/returns in a ParallelQuery?

void TestMethod(ICollection<int> coln)
{
    var result = AMehod(coln.AsParallel()).Select(LongRunningFunction).Sum();
}

IEnumerable<int> AMethod(IEnumerable<int> param)
{
    return param.Select(func());
}

int LongRunningFunction(T funcion)
{
   // do complex database query and return the results
}

Solution

  • You are right in that AMethod will use LINQ to objects (no parallelism). You indeed need two specialized functions because L2O and PLinq use different static query methods (Enumerable vs. ParallelEnumerable). (Or you make use of Reflection which I think is too much here.)

    You can make this work by calling the adapter method AsParallel in AMethod. AsParallel checks to see whether the argument is a parallel query and if yes it directly makes use of it.