Search code examples
powershellurlurlencodewebrequest

Powershell - Invoke-WebRequest to a URL with literal '/' (%2F) in it


I have been trying to access a URL with a / character in it from powershell, using the following command (it's a query to a gitlab server to retrieve a project called "foo/bar"):

Invoke-WebRequest https://server.com/api/v3/projects/foo%2Fbar -Verbose

Now, the odd thing is that using the PowerShell ISE or Visual Studio, the request is OK. When using PowerShell itself, the URL is automatically un-escaped and the request fails. E.g.

In ISE/VS:

$> Invoke-WebRequest https://server.com/api/v3/projects/foo%2Fbar -Verbose
VERBOSE: GET https://server.com/api/v3/projects/foo%2Fbar with 0-byte payload
VERBOSE: received 19903-byte response of content type application/json

StatusCode        : 200
StatusDescription : OK
Content           : .... data ....

In Powershell:

$> Invoke-WebRequest https://server.com/api/v3/projects/foo%2Fbar -Verbose
VERBOSE: GET https://server.com/api/v3/projects/foo/bar with 0-byte payload
Invoke-WebRequest : {"error":"404 Not Found"}
At line:1 char:1
+ Invoke-WebRequest 'https://server.com/api/v3/projects/foo%2Fbar ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

I have tried adding single and double quotes around the URL, but nothing is helping.

What could be the reason for this behaviour, and how do I make PS not un-escape the URL string?


Environment: Windows 7, also tested on Server 2012R2 with same results.

$> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      4.0
WSManStackVersion              3.0
SerializationVersion           1.1.0.1
CLRVersion                     4.0.30319.42000
BuildVersion                   6.3.9600.16406
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0}
PSRemotingProtocolVersion      2.2

Solution

  • Try the URL through this function

    function fixuri($uri){
      $UnEscapeDotsAndSlashes = 0x2000000;
      $SimpleUserSyntax = 0x20000;
    
      $type = $uri.GetType();
      $fieldInfo = $type.GetField("m_Syntax", ([System.Reflection.BindingFlags]::Instance -bor [System.Reflection.BindingFlags]::NonPublic));
    
      $uriParser = $fieldInfo.GetValue($uri);
      $typeUriParser = $uriParser.GetType().BaseType;
    $fieldInfo = $typeUriParser.GetField("m_Flags", ([System.Reflection.BindingFlags]::Instance -bor [System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::FlattenHierarchy));
    $uriSyntaxFlags = $fieldInfo.GetValue($uriParser);
    
    $uriSyntaxFlags = $uriSyntaxFlags -band (-bnot $UnEscapeDotsAndSlashes);
    $uriSyntaxFlags = $uriSyntaxFlags -band (-bnot $SimpleUserSyntax);
    $fieldInfo.SetValue($uriParser, $uriSyntaxFlags);
    }
    
    $uri = New-Object System.Uri -ArgumentList ("https://server.com/api/v3/projects/foo%2Fbar")
    fixuri $uri