Start-Job get the results to CSV

Question

I have the following PowerShell (v5) code and I want to write the results of the job to file. In the first line I want the headers, in the next lines I want only the results.

$siteToCheck = "google.com","stackoverflow.com","codeoverflow.com"
$alljobs     = $true
$fistLine    = $true

function AsyncJobs([array]$siteADDRs) {
    $sb = {
        Param($site)
        $baseSite = 'https://api.ssllabs.com/api/v3/analyze?host='
        $siteParams = '&publish=off'
        $fullSite = [String]::Format("{0}{1}{2}", $baseSite, $site, $siteParams)
        $results = Invoke-WebRequest -Uri $fullSite | ConvertFrom-Json
        while ($results.status -ne "READY" -and $results.status -ne "ERROR" ) {
            sleep 30
            $results = Invoke-WebRequest -Uri $fullSite | ConvertFrom-Json
        }
        (($results | select endpoints).endpoints) |
            ft @{Label="Site_Address"; exp={$site}}, grade, ipAddress
    }

    foreach ($site in $siteADDRs) {
        Start-Job -Name $site -ScriptBlock $sb -ArgumentList $site | Out-Null
    }
}

AsyncJobs -siteADDR $siteToCheck

while ($alljobs -eq $true) {
    $ourJobs = Get-Job
    foreach ($job in $ourJobs) {
        $jobResults = $null
        switch ($job.State) {
            {$_ -eq 'Running'} {
                Write-Host $job.Name " is still in runing sate"
            }
            {$_ -eq 'Completed'} {
                $jobResults = Receive-Job -Id $job.Id
                if ($fistLine -eq $true) {
                     ($jobResults | Out-String)
                     $fistLine = $false
                 } else {
                     $tmpR = ($jobResults | Out-String)
                 }
                 Remove-Job -Id $job.Id
            }
            {$_ -eq 'Failed'} {}
        }
    }
    $ourJobs = $null
    $ourJobs = Get-Job

    if ($ourJobs) {
        $alljobs = $true
    } else {
        $alljobs = $false
    }

    Start-Sleep -Seconds 10
}

Answer 1

In your scriptblock change

(($results | select endpoints).endpoints) |
    ft @{Label="Site_Address"; exp={$site}}, grade, ipAddress

to

$results | Select-Object -Expand endpoints |
    Select-Object @{Label="Site_Address"; exp={$site}}, grade, ipAddress

so that you get the output as custom objects rather than the formatting objects that Format-Table produces. You should generally avoid using Format-* cmdlet whenever further processing of your data is required. The cmdlets are only useful when displaying data directly to a user.

Export the data of completed jobs like this:

while (Get-Job -State 'Running') {
    $jobs = Get-Job -State 'Completed'
    $jobs | Receive-Job | Export-Csv 'output.csv' -NoType -Append
    $jobs | Remove-Job
    Start-Sleep -Milliseconds 200
}

After the loop finishes do a final export of the remaining completed jobs, then report and remove failed jobs:

$jobs = Get-Job -State 'Completed'
$jobs | Receive-Job | Export-Csv 'output.csv' -NoType -Append
$jobs | Remove-Job

Get-Job
Get-Job | Remove-Job