Improve fetching tasks so there are no multiple enumerations

Question

I have a do while which looks like this:

var tasks = await _JobProvider.FetchPendingTasks();

    do
    {
      foreach (var task in tasks)
      {
        await ExecuteStep(task);
      }

      tasks = await _JobProvider.FetchPendingTasks();
    }
    while (tasks.Any());

I fetch the pending task and it for example returns me one task inside a IEnumerable. Inside await ExecuteStep(task); more tasks could be generated. this is why i am doing the second tasks = await _JobProvider.FetchPendingTasks();. With it I check if new task were generated inside await ExecuteStep(task); and if there are it does the do while loop again. This causes a mutiple enumeration of tasks which I think could be improved.

Is there a way to improve this code so there are no more multiple enumerations?

Answer 1

You could use a Queue<Task>, and enumerate it endlessly with a while loop instead of foreach:

var queue = new Queue<Task>(await _JobProvider.FetchPendingTasks());
while (queue.Count > 0)
{
    var task = queue.Dequeue();
    Task[] moreTasks = await ExecuteStep(task);
    foreach (var t in moreTasks) queue.Enqueue(t);
}

This assumes the Queue<Task> will not be accessed concurrently by multiple threads. Otherwise you could use a ConcurrentQueue<Task>.