Search code examples
c#parallel-processingasync-awaitparallel.foreach

Must we wait for all async calls when using Parallel.Foreach?


Assuming we have a List of filters, is it ok to use Parallel.ForEach to make async calls and then use them? Must we have to wait for all calls to end? What if we have 3 async calls and one fails/times out?

var results = new ConcurrentBag<Service>();

Parallel.ForEach(filters, async filter =>
{
   var result = await serviceClient.GetAllAsync(filter).ConfigureAwait(false);

   results.AddRange(result);
});

MyMapper.Map(results);

where MyMapper has a method:

Map(IEnumerable<Service> services) {
   foreach(Service service in services) {
      //do stuff
   }
}

Solution

  • The Parallel library is thought to process millions of items in parallel. You won't get any improvement using Parallel.ForEach there. You could use Task.WhenAll instead.

    var tasks = new List<Task>();
    
    foreach(var filter in filters)
    {
       tasks.Add(serviceClient.GetAllAsync(filter));
    }
    
    results.AddRange(await Task.WhenAll(tasks).ConfigureAwait(false));
    

    You could do some LinQ. I didn't so the idea is more clear. With this code you could have some degree of parallelism and the await will help you with exceptions.