How to download a webstring from multiple web-servers in parallel via Powershell?

Question

I need to download some webcontent from many servers in parallel as part of a scheduled job, but I cannot find a correct way to run the download in parallel/async. How can this be done?

Without any parallelism I can do it this way, but it is very slow:

$web = [System.Net.WebClient]::new()
[Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

# $srvList is a list of servers of viariable length
$allData = ""
foreach ($srv in $srvList) {
    $url = "https:\\$srv\MyWebPage"
    $data = $web.DownloadString($url)
    $allData += $data
}

But how to do this in parallel via "$web.DownloadStringAsync"? I found this snippet, but I dont see how to get the result of each call and how to concatenate it:

$job = Register-ObjectEvent -InputObject $web -EventName DownloadStringCompleted -Action {
    Write-Host 'Download completed'
    write-host $EventArgs.Result
}
$web.DownloadString($url)

Does someone know, how to get this solved in a short & smart way?

Answer 1

Finally I found a simple solution via events. Here is my code-snippet:

cls
Remove-Variable * -ea 0

[System.Net.ServicePointManager]::ServerCertificateValidationCallback = $null
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

$srvList = @('srv1','srv2','srv3')
$webObjList = [System.Collections.ArrayList]::new()
$eventList  = [System.Collections.ArrayList]::new()
$resultList = [System.Collections.ArrayList]::new()

$i=0
foreach ($srv in $srvList) {
    $null  = $webObjList.add([System.Net.WebClient]::new())
    $null  = $eventList.add($(Register-ObjectEvent -InputObject $webObjList[$i] -EventName DownloadStringCompleted -SourceIdentifier $srv))
    $null  = $resultList.add($webObjList[$i].DownloadStringTaskAsync("https://$srv/MyWebPage"))
    $i++
}
do {sleep -Milliseconds 10} until ($resultList.IsCompleted -notcontains $false)
foreach ($srv in $srvList) {Unregister-Event $srv}

# show all Results:
$resultList.result