VB.Net Threading running a "rolling barrel"

Question

Might not have explained it well in the title, but...

I have a number of things I'm running as tasks. However, due to memory limitations, I'm batching them up:

        Dim fullCount As Integer = tbl.Rows.Count - 1
        If fullCount <= _batchsize Then
            For Each dr As DataRow In tbl.Rows 
                Dim threadData As processThreadData = New processThreadData
                threadData.code = stuff
                threadData.otherStuff = more stuff
                task = New Task(Function() MyFunction(threadData))
                task.Start()
                tasks.Add(task)
                Thread.Sleep(0)
            Next
            task.WaitAll(tasks.ToArray)
        Else

            Dim batches As Integer = fullCount \ _batchsize
            If (fullCount / _batchsize) <> batches Then
                batches += 1
            End If
            Dim batchcount As Integer = 0
            Dim runningThreads As Integer = 0
            For Each dr As DataRow In tbl.Rows
                Dim threadData As processThreadData = New processThreadData
                threadData.code = stuff
                threadData.otherStuff = more stuff
                task = New Task(Function() MyFunction(threadData))
                task.Start()
                tasks.Add(task)
                Thread.Sleep(0)
                If runningThreads < _batchsize Then
                    'add another 
                    runningThreads += 1
                Else
                    batchcount += 1
                    Debug.Print(runningThreads.ToString + "/" + batchcount.ToString)
                    task.WaitAll(tasks.ToArray)
                    runningThreads = 0
                End If
            Next
            If batchcount <= batches Then
                task.WaitAll(tasks.ToArray)
                runningThreads = 0
            End If
        End If

This works well enough, but if one of the processes takes a long time (as can happen) it slows down the whole batch until that one has finished, then another batch of _batchsize number of threads can start.

What I'd like to do is have a max number of threads that can be running at any one time (_batchsize = 50 as an arbitrary number) and then when one of those finishes, another starts so there will always be 50 running, until the last 50, of course!

Is there a way to do this in VB.Net?

Answer 1

See the section on "Throttling" in the .NET docs on the task-based asynchronous pattern: https://learn.microsoft.com/en-us/dotnet/standard/asynchronous-programming-patterns/consuming-the-task-based-asynchronous-pattern

In short, instead of waiting on all of the tasks in a batch (and then retiring the entire batch), you put a subset into a collection, wait on any to finish, and replace the finished task with a new one.