Problems implementing properly a Parallel.ForΕach loop and continue other work in the mean time

Question

We are using the .Net Framework 4.8.2. For some part of the app I want to execute many long running CPU and IO bound processes (illustrated by Thread.Sleep()) in parallel. That's why I use the Parallel.ForΕach loop. These results are not needed for some time. That's why I want to continue with different work and "suspend" this task of tasks (so to speak) to a different thread whereas each Parallel.ForΕach is executed again on a different thread. That's why I return the Parallel.ForΕach encapsulated in a Task.Run() statement. The code looks as follows:

internal class Program
{
    static async Task Main(string[] args)
    {
        Console.WriteLine($"Before ParallelForeach on thread: {Environment.CurrentManagedThreadId}"); //Main debug 1
        await TaskContainingParallelForeach();
        Console.WriteLine($"After  ParallelForeach on thread: {Environment.CurrentManagedThreadId}"); //Main debug 2
        int counter = 0;
        while (counter < 7)
        {
            Console.WriteLine($"{counter}. one second gone on thread:  {Environment.CurrentManagedThreadId} at {DateTimeOffset.Now.ToUnixTimeMilliseconds()}");
            Thread.Sleep(1000);
            counter++;
        }

        Console.ReadKey();
    }

    public static Task TaskContainingParallelForeach()
    {
        return Task.Run(() =>
        {
            Console.WriteLine($"Inside ParallelForeach on thread: {Environment.CurrentManagedThreadId}");
            int[] numbers = { 1, 3, 2 };

            Parallel.ForEach(numbers, (num) =>
            {
                Console.WriteLine($"{num} is running on thread: {Environment.CurrentManagedThreadId}");
                Thread.Sleep(num * 1000);
                Console.WriteLine($"{num}    stopped on thread: {Environment.CurrentManagedThreadId}        at {DateTimeOffset.Now.ToUnixTimeMilliseconds()}");
            });
        });
    }}

Which gives me the following output:

Console Output without await:
Before TaskContainingParallelForeach on thread: 1
Inside TaskContainingParallelForeach on thread: 3  // why is -below- first done all the work from the ParallelForeach? 
1 is running on thread: 3
3 is running on thread: 4
2 is running on thread: 6
1    stopped on thread: 3        at 1734365740995
2    stopped on thread: 6        at 1734365741996
3    stopped on thread: 4        at 1734365742996
After  TaskContainingParallelForeach on thread: 3 // why is it the thread with id 3 here - expected was 1
0. one second gone on thread:  3 at 1734365742996
1. one second gone on thread:  3 at 1734365743997
2. one second gone on thread:  3 at 1734365745008
3. one second gone on thread:  3 at 1734365746023
4. one second gone on thread:  3 at 1734365747035
5. one second gone on thread:  3 at 1734365748047
6. one second gone on thread:  3 at 1734365749061

The Parallel.ForEach is run and only after it finished - the while loop starts. The await-operator should behave like: (Original docs)The await operator suspends evaluation of the enclosing async method until the asynchronous operation. My async method here is the TaskContainingParallelForeach. Or is it not?

And why is the thread id for After TaskContainingParallelForeach on thread 3 and not 1? And why is the while loop not beeing executed on thread 1? However, removing the await operator from await TaskContainingParallelForeach(); gets the desired result:

Console output without await:
Before TaskContainingParallelForeach on thread: 1
After  TaskContainingParallelForeach on thread: 1
Inside TaskContainingParallelForeach on thread: 3
1 is running on thread: 3
3 is running on thread: 4
2 is running on thread: 5
0. one second gone on thread:  1 at 1734364096009
1    stopped on thread: 3        at 1734364097021
1. one second gone on thread:  1 at 1734364097036
2    stopped on thread: 5        at 1734364098027
2. one second gone on thread:  1 at 1734364098043
3    stopped on thread: 4        at 1734364099022
3. one second gone on thread:  1 at 1734364099053
4. one second gone on thread:  1 at 1734364100064
5. one second gone on thread:  1 at 1734364101077
6. one second gone on thread:  1 at 1734364102078

This looks great!

1. all work The Parallel.Foreach loop is started
2. In the mean-time the while loop started its work
3. The whileloop is on the expected thread 1.

But thats not how i think await with Task.Run() and Parallel.Foreach should be used together. How can i iprove this code?

Thanks in advance!

Answer 1

I think what you want to do is this:

Console.WriteLine($"Before ParallelForeach on thread: {Environment.CurrentManagedThreadId}"); //Main debug 1
var parallelTask =  TaskContainingParallelForeach();
Console.WriteLine($"After  ParallelForeach on thread: {Environment.CurrentManagedThreadId}"); //Main debug 2
int counter = 0;
while (counter < 7) {
    Console.WriteLine($"{counter}. one second gone on thread:  {Environment.CurrentManagedThreadId} at {DateTimeOffset.Now.ToUnixTimeMilliseconds()}");
    Thread.Sleep(1000);
    counter++;
}

await parallelTask; // want to join here