Search code examples
powershellfreezeinvoke-webrequest

Invoke-WebRequest freezes / hangs


Why does the cmdlet "Invoke-WebRequest" freezes / hangs on certains URLs? Any workaround possible? I want to access the "span" objects for a given webpage and this cmdlet would be very useful if it didn't hang like that.

For example, this hangs:

Invoke-WebRequest -Uri "https://cloud.google.com/chrome-enterprise/browser/download/"

This doesn't :

Invoke-WebRequest -Uri "https://www.microsoft.com/fr-ca/"

-UseBasicParsing makes it run but I want to use the functionality of what Invoke-WebRequest returns without basic parsing because with basic parsing, the span field i'm trying to extract is not populated.


Solution

  • This seems to still be a bug in powershell

    https://github.com/PowerShell/PowerShell/issues/2867

    A possible work around is to convert the item into parsed HTML manually

    Function ConvertTo-NormalHTML {
        param([Parameter(Mandatory = $true, ValueFromPipeline = $true)]$HTML)
    
        $NormalHTML = New-Object -Com "HTMLFile"
        $NormalHTML.IHTMLDocument2_write($HTML.RawContent)
        return $NormalHTML
    }
    
    $Content = (Invoke-WebRequest -Uri "https://cloud.google.com/chrome-enterprise/browser/download" -UseBasicParsing ).Content
    
    $ParsedHTML = ConvertTo-NormalHTML -HTML $Content
    
    $ParsedHTML