Run N parallel jobs in powershell

Question

I have the following powershell script

$list = invoke-sqlcmd 'exec getOneMillionRows' -Server...
$list | % {
    GetData $_ > $_.txt
    ZipTheFile $_.txt $_.txt.zip
    ...
}

How to run the script block ({ GetDatta $_ > $_.txt ....}) in parallel with limited maximum number of job, e.g. at most 8 files can be generated at one time?

Answer 1

The Start-Job cmdlet allows you to run code in the background. To do what you'd ask, something like the code below should work.

foreach ($server in $servers) {
    $running = @(Get-Job | Where-Object { $_.State -eq 'Running' })
    if ($running.Count -le 8) {
        Start-Job {
             Add-PSSnapin SQL
             $list = invoke-sqlcmd 'exec getOneMillionRows' -Server...
             ...
        }
    } else {
         $running | Wait-Job
    }
    Get-Job | Receive-Job
}

Hope this helps.