Running multiple sets of parallel threads using Parallel.ForEach

Question

I'm trying to run two sets of parallel tasks. My task.Invoke() method is async, so in the first set of tasks, I leave off .Wait(); but in the second set, I use .Wait() so it won't exit the parallel set until all the tasks are actually complete.

This generally works in that the A group tasks start and then the B group tasks start, all the tasks work in parallel within their group with group A running will group B runs and group b doesn't exit until all tasks of its tasks are done.

var parallelA = new List<IMyTask>();
parallelA.Add(new MyTask());
parallelA.Add(new MyTask());

await Task.Run(() => Parallel.ForEach(parallelA, task =>
{
    task.Invoke();
    }));

var parallelB = new List<IMyTask>();
parallelB.Add(new MyTask());
parallelB.Add(new MyTask());

await Task.Run(() => Parallel.ForEach(parallelB, task =>
{
    task.Invoke().Wait();
}));

The problem is that the group a tasks don't necessarily complete before the task b group is done so the function exits while group a is still running in some cases.

I imagine I could re-write it so that they all interoperate, possibly by having parallel loop of parallel loops using a collection of collection of tasks, but is that going to have some detrimental impact on the thread management, etc inside the clr?

The background is that I have a background process that runs on interval, I break up the tasks it runs into two groups, group A is a couple of long running tasks and group B is a series of short running tasks stored in a queue. Both sets have equal priority with the same start and stop time with the queue able to just be stopped when the time is up and any tasks can restart next cycle.

Answer 1

You can just use WhenAll

var parallelA = new List<IMyTask>();
parallelA.Add(new MyTask());
parallelA.Add(new MyTask());

var task1 = Task.Run(() => Parallel.ForEach(parallelA, task =>
   {
      task.Invoke();
   }));

var parallelB = new List<IMyTask>();
parallelB.Add(new MyTask());
parallelB.Add(new MyTask());

var task2 = Task.Run(() => Parallel.ForEach(parallelB, task =>
   {
      task.Invoke().Wait();
   }));

await Task.WhenAll(task1, task2);

Note, there are probably better ways to do this though, you could just use lists of task and not use Parallel.ForEach at all, though you are using an IMyTask interface and its hard to infer whats going on