Powershell How to integret 4 script block in loop, as Start-Rsjob to run parallel & get each's status?

Question

Simple question, but its solution will become life-Saver for me.

I have powershell script as below that just copy paste some source folders to/on some destination locations as routeen as weekly backup.

Source folders are on windows Server2K12 on HDD with multi core processor & destinations on directly connected removable SSD as backup drive.

Window's copy-paste is very painfull as stuck inbetween operation, & mostly stop incomplete.

Through Powershell , can get job done even on any file/folder attribute with it's powerful -force - recurse parameters.

But i can't integrate that 4 script blocks , which require to run in parallel ,so, in loop with Start-Rs job. (I choose RSjob, because in POwershell-ISE there are no any Foreach-Parallel cmdlet available, its available only in Visual Studio Code & in PWSH)

Code:-

 
 $THROTT=[INT]$env:NUMBER_OF_PROCESSORS + 1
  
 Start-RSJob -ScriptBlock {COPY-ITEM -LiteralPath '\\?\D:\S1' -Destination '\\?\F:\200223\D1\' -Force -Recurse} -Throttle $THROTT 
 Start-RSJob -ScriptBlock {COPY-ITEM -LiteralPath '\\?\D:\S2' -Destination '\\?\F:\200223\D2\' -Force -Recurse} -Throttle $THROTT 
 Start-RSJob -ScriptBlock {COPY-ITEM -LiteralPath '\\?\D:\S3' -Destination '\\?\F:\200223\D3\' -Force -Recurse} -Throttle $THROTT
 Start-RSJob -ScriptBlock {COPY-ITEM -LiteralPath '\\?\D:\S4' -Destination '\\?\F:\200223\D4\' -Force -Recurse} -Throttle $THROTT

GET-RSJOB |FOREACH {WHILE ($_.State -eq 'RUNNING') {START-SLEEP 1 ;Write-Host $($_.NAME) ' ' 'RUNNING'}}

I already study this link powershell run script in parallel but its different, it is about how to take sources/targets for jobs from other source , like from file as via use of Get-Content cmdlet. & It's Great feature Parallel Processing. To dramatically reduce processing time as per Link here.

https://devblogs.microsoft.com/powershell/powershell-foreach-object-parallel-feature/

But my current requirement is direct hard code sources & destinations in current script as they (mostly destinations) often change periodically & I Can not integrate that approach (Sources/Destinations written in/from file) in my current requirement.

My current script try to check each job's status , but result display only for first job only, not for all-job's.

If i can construct all jobs in single loop, then i can check each job's status in loop after some time interval like on every 30 seconds.

I already try construct each block in variable like

$job1 = script-block {...} $job2=script-block {...} etc. But can't success that integrate in loop.

I use powershell-ISE , If, there are solution found, then how to run that script with/through PWSH (Powershell-Console)?

Can I use pwsh's foreach-parallel?, which changes require change in script to run through pwsh?

My sources folders are mostly constants , so i can directly provide only required destinations to script without open , write, & save script for writing changing destinations each time.

Thank you.

Answer 1

If you're willing to give the function of this answer a try or, preferably, the one in the GitHub repo which has a different code-base and added features, this is how the code would look. It already handles pipeline processing, similar to ForEach-Object -Parallel but compatible with Windows PowerShell 5.1.

@{ LiteralPath = '\\?\D:\S1'; Destination = '\\?\F:\200223\D1\' },
@{ LiteralPath = '\\?\D:\S2'; Destination = '\\?\F:\200223\D2\' },
@{ LiteralPath = '\\?\D:\S3'; Destination = '\\?\F:\200223\D3\' },
@{ LiteralPath = '\\?\D:\S4'; Destination = '\\?\F:\200223\D4\' } |
    Invoke-Parallel { Copy-Item @_ -Force -Recurse } -ThrottleLimit ($env:NUMBER_OF_PROCESSORS + 1)

If you want to install the repo version, you can get it through the PowerShell Gallery:

Install-Module -Name PSParallelPipeline -Scope CurrentUser