Search code examples
c#.netasynchronousparallel.foreach

How can I limit Parallel.ForEach?


I have a Parallel.ForEach() async loop with which I download some webpages. My bandwidth is limited so I can download only x pages per time but Parallel.ForEach executes whole list of desired webpages.

Is there a way to limit thread number or any other limiter while running Parallel.ForEach?

Demo code:

Parallel.ForEach(listOfWebpages, webpage => {
  Download(webpage);
});

The real task has nothing to do with webpages, so creative web crawling solutions won't help.


Solution

  • You can specify a MaxDegreeOfParallelism in a ParallelOptions parameter:

    Parallel.ForEach(
        listOfWebpages,
        new ParallelOptions { MaxDegreeOfParallelism = 4 },
        webpage => { Download(webpage); }
    );
    

    MSDN: Parallel.ForEach

    MSDN: ParallelOptions.MaxDegreeOfParallelism