Use Semaphore and Register-ObjectEvent to throttle the number of processes, where is the problem?

Question

Below is my code. It basically uses a Semaphore and and Register-ObjectEvent to throttle the number of ffmpeg processes. I have a lot of video files, but at anytime, only two of them should being processed.

$working_dir = "C:\temp\video\test"
$input_files = @(Get-ChildItem -Path $working_dir -Include @("*.webm", "*.mkv") -Recurse -File)

$throttleLimit = 2
$semaphore = [System.Threading.Semaphore]::new($throttleLimit, $throttleLimit)
$jobs = @()

foreach ($v in $input_files) {
    $new_name = [regex]::replace($v.fullname, '\.[^\.]+$', '') + ".mp4"
    $ArgumentList = "-i `"$($v.fullname)`" -metadata comment= -metadata title= -filter_complex `"drawtext=fontsize=10:fontfile='I\:\\temp\\sarasa-term-sc-bold.ttf':text='':x=228:y=840:fontcolor=000000`" -c:a copy -y `"$($new_name)`""

    $semaphore.WaitOne()

    $process = Start-Process -FilePath "ffmpeg.exe" -ArgumentList $ArgumentList -PassThru -NoNewWindow

    $job = Register-ObjectEvent -InputObject $process -EventName "Exited" -Action {
        $semaphore.Release()
        Unregister-Event $eventSubscriber.SourceIdentifier
        Remove-Job $eventSubscriber.Action
    }

    $jobs += $job
}

$jobs | Wait-Job | Receive-Job

The problem I noticed is that, only two files got processed by ffmpeg. The PowerShell console that I pasted the upper code snippet looks like this when the first two video files got processed.

It looks like ffmpeg didn't exit. But when I check from another PowerShell session, no ffmpeg process was found.

PS C:\WINDOWS\system32> ps ffmpeg
ps : Cannot find a process with the name "ffmpeg". Verify the process name and call the cmdlet again.
At line:1 char:1
+ ps ffmpeg
+ ~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (ffmpeg:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

PS C:\WINDOWS\system32> ps *ffmpeg*
PS C:\WINDOWS\system32>

Where is the problem? Is the logic in my code correct?

Answer 1

Your code has 2 problems:

1. The Action block does not know what $semaphore is, you need to pass the reference of this instance with -MessageData and call it back using $event.MessageData.

2. $semaphore.WaitOne() will lock the thread indefinitely, WaitOne() won't allow PowerShell to check for interrupts. Basically you need to change this call for a loop with a timeout.

Here is a simple working example, throttling the number of notepad processes that can be opened at the same time. I have also changed your Semaphore for a better version of it.

$lock = [System.Threading.SemaphoreSlim]::new(2, 2)
$jobs = foreach($i in 0..10) {
    while(-not $lock.Wait(200)) { }
    $proc = Start-Process notepad -PassThru

    $registerObjectEventSplat = @{
        InputObject = $proc
        EventName   = 'Exited'
        MessageData = $lock
        Action      = {
            $event.MessageData[0].Release()
            Unregister-Event $eventSubscriber.SourceIdentifier
            Remove-Job $eventSubscriber.Action
        }
    }
    Register-ObjectEvent @registerObjectEventSplat
}

A much easier approach would be using the ThreadJob Module available through the Gallery for PowerShell 5.1 and pre-installed in newer versions of PowerShell 7+. The Start-ThreadJob cmdlet has a built-in throttling mechanism that simplifies the above process a lot. For comparison:

$jobs = foreach($i in 0..10) {
    Start-ThreadJob {
        Start-Process notepad -Wait
    } -ThrottleLimit 2
}
$jobs | Receive-Job -AutoRemoveJob -Wait