Limit number of Start-Process running in powershell

Question

I have tried to limit the number of Start-Process running from a Powershell, but I can't seem to get it to work.

I tried to follow this process: https://exchange12rocks.org/2015/05/24/how-to-limit-a-number-of-powershell-jobs-running-simultaneously/ and Run N parallel jobs in powershell

But these are for Jobs not Processes, and I would like to remove the -Wait from the Start-Process

My concern with the script is that if there are 1000 audio files in the folder, then FFMpeg would crash the system.

---

# get the folder for conversion
function mbAudioConvert {
    [Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms") | Out-Null
    [System.Windows.Forms.Application]::EnableVisualStyles()

    $fileBrowser = New-Object System.Windows.Forms.FolderBrowserDialog
    $fileBrowser.SelectedPath = "B:\"
    $fileBrowser.ShowNewFolderButton = $false
    $fileBrowser.Description = "Select the folder with the audio which you wish to convert to Avid DNxHD 120 25P 48kHz"

    $mbLoop     = $true
    $mbCount    = 0001
    $mbMaxJob   = 4

    while( $mbLoop ) {
        if( $fileBrowser.ShowDialog() -eq "OK" ) {
            $mbLoop     = $false


            $mbImage    = ( Get-Item -Path "C:\Users\user\Desktop\lib\AudioOnly.jpg" )
            $mbff32     = ( Get-Item -Path "C:\Users\user\Desktop\lib\ffmpeg32.exe" )
            $mbff64     = ( Get-Item -Path "C:\Users\user\Desktop\lib\ffmpeg64.exe" )

            $mbFolder   = $fileBrowser.SelectedPath
            $mbItemInc  = ( ls $mbFolder\* -Include *.mp3, *.MP3, *.wav*, *.WAV*, *.ogg, *.OGG, *.wma, *.WMA, *.flac, *.FLAC, *.m4a, *.M4a )
            $mbProgress = ( Get-ChildItem -Path $mbItemInc )

            $mbHasRaw   = ( $mbFolder + "\RAW" )

            if( !( Test-Path -Path $mbHasRaw ) ) {
                # force create a RAW folder if it does not exist
                New-Item -ItemType Directory -Force -Path "$mbHasRaw"
            }


            foreach( $mbItem in $mbItemInc ) {

                $mbCheck    = $false

                # output the progress
                # Suggestion: You might want to consider updating this after starting the job and do the final update after running ex. Get-Job | Wait-Job to make the progress-bar stay until all processes are finished
                #Write-Progress -Activity "Counting files for conversion" -status "Currently processing: $mbCount" -percentComplete ($mbCount / $mbItemInc.count*100)

                # limit the run number
                while ($mbCheck -eq $false) {

                    if( (Get-Job -State 'Running').count -lt $mbMaxJob) {

                        $mbScriptBlock = {
                            $mbItemName = $using:mbItem.BaseName

                            $mbNewItem  = ( $using:mbFolder + "\RAW\" + $mbItemName + ".mov" )
                            $mbArgs     = " -loop 1 -i $using:mbImage -i $using:mbItem -shortest -c:v dnxhd -b:v 120M -s 1920x1080 -pix_fmt yuv422p -r 25 -c:a pcm_s16le -ar 48k -af loudnorm=I=-12 $mbNewItem"

                            Start-Process -FilePath $using:mbff32 -ArgumentList $mbArgs -NoNewWindow -Wait
                        }

                        Start-Job -ScriptBlock $mbScriptBlock

                        #The job-thread doesn't know about $mbCount, better to increment it after starting the job
                        $mbCount++
                        $mbCheck  = $true          
                    }

                }
            }

        } else {

            $mbResponse = [System.Windows.Forms.MessageBox]::Show("You have exited out of the automation process!", "User has cancelled")
            if( $mbResponse -eq "OK" ) {
                return
            }
        }
    }

    $fileBrowser.SelectedPath
    $fileBrowser.Dispose()
}

# call to function
mbAudioConvert

Answer 1

1. You edit $mbCheck, but the while loop is testing $Check which means the while-loop will never execute as $Check -eq $false is $false when $Check is not defined
2. Variables created outside the job script-block needs to be passed as an argument or you need to use the using: variable-scope to pass them in (PowerShell 3.0 or later). Added it to $mbItem, $mbff32, $mbImage and $mbFolder in the example which were not defined.
3. $mbMaxJob is not defined. The get running jobs-check will never be true and no processes will start
4. $mbCount not defined. Progress bar won't work
5. echo "$mbCount. $mbNewItem" won't return anything unless you use Receive-Job at some point to get the output from a job

Try:

#DemoValues
$mbItemInc = 1..10 | % { New-Item -ItemType File -Name "File$_.txt" }
$mbff32 = "something32"
$mbFolder = "c:\FooFolder"
$mbImage = "BarImage"
$mbMaxJob = 2
$mbCount = 0

foreach( $mbItem in $mbItemInc ) {

    $mbCheck    = $false

    # output the progress
    # Suggestion: You might want to consider updating this after starting the job and do the final update after running ex. Get-Job | Wait-Job to make the progress-bar stay until all processes are finished
    Write-Progress -Activity "Counting files for conversion" -status "Currently processing: $mbCount" -percentComplete ($mbCount / $mbItemInc.count*100)

    # limit the run number
    while ($mbCheck -eq $false) {

        if ((Get-Job -State 'Running').count -lt $mbMaxJob) {

            $mbScriptBlock = {

                 Param($mbItem, $mbFolder, $mbImage, $mbff32)
                #Filename without extension is already available in a FileInfo-object using the BaseName-property
                $mbItemName = $mbItem.BaseName

                $mbNewItem  = ( $mbFolder + "\RAW\" + $mbItemName + ".mov" )
                $mbArgs     = "-loop 1 -i $mbImage -i $mbItem -shortest -c:v dnxhd -b:v 120M -s 1920x1080 -pix_fmt yuv422p -r 25 -c:a pcm_s16le -ar 48k -af loudnorm=I=-12 $mbNewItem"

                Start-Process -FilePath $mbff32 -ArgumentList $mbArgs -NoNewWindow -Wait
            }

            Start-Job -ScriptBlock $mbScriptBlock -ArgumentList $mbItem, $mbFolder, $mbImage, $mbff32

            #The job-thread doesn't know about $mbCount, better to increment it after starting the job
            $mbCount++
            $mbCheck  = $true          
        }

    }
}