ManualResetEvent is not waiting for threadpool completion

Question

I have list of batches to process. forever.
I want to do each chunk (5) in parallel, and when it is done move to the next chunk.
for some reason, the code bellow is not waiting for the chunk to be done and continue even if it is not completed.

while (true)
{
    foreach (string[] urlsArr in chunks)
    { 
        int i = 0;
        foreach (var url in urlsArr)
        {
            ThreadPool.QueueUserWorkItem(x =>
            {
                ProccessUrl(url, config, drivers[i]);
                _resetEvent.Set();
                i++;
            });
        }
        _resetEvent.WaitOne();// this is not really waiting.
    }
}

Answer 1

Here is a version with Tasks(async/await)

while (true)
        {
            foreach (string[] urlsArr in chunks)
            {
                Task[] tasks = new Task[urlsArr.Length];
                for (int i = 0; i < urlsArr.Length; i++)
                {
                    var url = urlsArr[i];
                    var driver = drivers[i];
                    tasks[i] = Task.Run(() => { ProccessUrl(url, config, driver); });
                }

                await Task.WhenAll(tasks);
            }
        }

note that it also fixes a problem with the 'i' variable in the original code which was not incremented in a thread safe way(could be fixed with Interlocked.Increment).

If your code is not async you can wait for the tasks to finish in the thread(but this is blocking) instead

Task.WhenAll(tasks).Wait();