Using PowerShell, can I copy one file from my local system to many remote computers in parallel?

Question

As the title suggests, I cant seem to get either workflows or foreach-object -Parallel to work.

Here is one attempt:

$Computers = Get-Content 'C:\temp\Inputfiles\computers.txt'
$SourcePath = 'C:\Temp\Inputfiles\VMTools\Setup64.exe'

$Job = $Computers | ForEach-Object -Parallel {
  Copy-Item -Path $using:SourcePath -Destination "\\$_\C$\Temp\" -Force
  Start-Sleep -Seconds 1
} -ThrottleLimit 5 -AsJob

$Job | Wait-Job | Receive-Job

I tried variations of the above, but cant get it to work.

Here is the error I was getting on the above:

>  .\Copy-FileToRemote.ps1
ForEach-Object : Parameter set cannot be resolved using the specified named parameters.
At C:\Temp\ITScript\Workflow\Copy-FileToRemote.ps1:4 char:21
+ $Job = $Computers | ForEach-Object -Parallel {
+                     ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : MetadataError: (:) [ForEach-Object], ParameterBindingException
    + FullyQualifiedErrorId : AmbiguousParameterSet,Microsoft.PowerShell.Commands.ForEachObjectCommand

Wait-Job : Cannot validate argument on parameter 'Job'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.
At C:\Temp\ITScript\Workflow\Copy-FileToRemote.ps1:9 char:8
+ $Job | Wait-Job | Receive-Job
+        ~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Wait-Job], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.WaitJobCommand

Answer 1

Preface:

• Windows PowerShell workflows were introduced in v3, but never really took off. Given that they're no longer available in PowerShell (Core) v6+, where all future development effort will go, they should be considered obsolete.

• In regular PowerShell sessions, the -Parallel parameter of the ForEach-Object cmdlet requires PowerShell (Core) 7+.

---

Your solution options are:

• Upgrade to PowerShell (Core) 7, in which case your code would work as-is, however:

  • The Start-Sleep -Seconds 1 isn't necessary and will only slow processing down.

  • You only need -AsJob (and the later Wait-Job and Receive-Job calls) if you don't want to wait for completion right away.

  • $Job | Wait-Job | Receive-Job can be simplified to $Job | Receive-Job -Wait -AutoRemoveJob, which additionally also deletes the jobs afterwards.

• If you're stuck with Windows PowerShell:

  • Either: Simply omit -Parallel (and -ThrottleLimit 5 and -AsJob), at the expense of then getting sequential (and synchronous) processing.

  • Or: If installing a module is an option, use the Start-ThreadJob cmdlet, which offers a lightweight, much faster thread-based alternative to the child-process-based regular background jobs created with Start-Job. It comes with PowerShell 7, but must be installed on demand in Windows PowerShell, e.g., Install-Module ThreadJob -Scope CurrentUser.

    • Note: In principle, you may alternatively use the built-in Start-Job cmdlet, but - given that each job runs in a child process rather than a thread - this will be much slower and more resource-intensive; also, unlike Start-ThreadJob, Start-Job doesn't support throttling, so a large number of target computers could overwhelm your system.

# Assumes that the ThreadJob module is available, which requires
# on-demand installation in Windows PowerShell.

$Computers = Get-Content 'C:\temp\Inputfiles\computers.txt'
$SourcePath = 'C:\Temp\Inputfiles\VMTools\Setup64.exe'

# Start a thread job for each computer.
$Job = $Computers | ForEach-Object {
  Start-ThreadJob -ThrottleLimit 5 {
    Copy-Item -Path $using:SourcePath -Destination "\\$using:_\C$\Temp\" -Force
  }
}

$Job | Receive-Job -Wait -AutoRemoveJob

Note how the caller's $_ value (the current pipeline input object) must in this case also be referenced via $using: - unlike with ForEach-Object -Parallel.

---

Note:

This answer originally showed an approach based on
Copy-Item -ToSession; however, the -ToSession approach is generally not recommended: ^{Thanks, noam.}

• Only a single session object may be passed to -ToSession, i.e. you can only target one remote computer at a time, so you get no parallelism as part of the command.

• -ToSession is much slower than SMB-based copying (targeting the remote computer via a file share and UNC path); quoting a team member from GitHub issue #14646 (emphasis added):

• -ToSession means it's going over PowerShell remoting which adds lots of overhead for simply transferring files. It's really only intended to be used if direct SMB is not available.