Search code examples
c#.netlinqparallel-processingplinq

.NET 4 Parallel.ForEach and PLINQ: can they overwhelm the thread pool and kill the app performance?


The more I use Parallel.ForEach and PLINQ in my code, the more faces and code review push backs I am getting. So I wonder is there any reason for me NOT to use PLINQ, at extreme, on each LINQ statement? Can the runtime not be smart enough to start spawning so many threads (or consuming so many threads from the thread pool) that the app performance would actually degrade instead of improve? The same question applies to Parallel library.

I do understand implications related to thread-safety and overhead of using multi-threading. I also realize not everything is good for parallelizing. All I am wondering about if I should stop defending my approaches and just give up on these two fine things because my peers think I'd better do thread control myself instead of relying on .NET facilities?

UPDATE: please assume the hardware is sufficiently good to satisfy prerequisites for use of multithreading.


Solution

  • To set the number of worker threads you can use .WithDegreeOfParallelism(N)

    eg

    var query = from item in source.AsParallel().WithDegreeOfParallelism(2)
                where Compute(item) > 42
                select item;
    

    See http://msdn.microsoft.com/en-us/library/dd997425.aspx