Search code examples
c#unicodewebrequest

WebRequest.Create improperly decodes url


I have a url like this: http://localhost:8080/search.json?q=L%u00e6r, which is the encoded search for Lær.

Unfortunately creating a WebRequest from this url using WebRequest.Create(url) produces the following url: http://localhost:8080/search.json?q=L%25u00e6r.

Notice that it incorrectly decodes %u00e6, and produces %25u00e6r. Is there a way to either convert this kind of unicode escaped value or get WebRequest.Create to properly handle it?

This should most likely be reported as a bug to the .net team. WebRequest.Create() cannot use the query string returned by Request.QueryString.ToString() if the query contains a §, æ, ø or å (or any other non-ascii character). Here is a small mvc action which can be used to test it. Call it with the query Query?q=L%C3A6r

public ActionResult Query()
{
    var query = Request.QueryString.ToString();
    var url = "http://localhost:8080?" + query;


    var request = WebRequest.Create(url);
    using (var response = request.GetResponse())
    using (var stream = response.GetResponseStream())
    {
        return new FileStreamResult(stream, "text/plain");
    }
}

Edit:

Unfortunately @animaonline's solution does not work with urls like http://localhost:8080/search.json?q=Lek+%26+L%u00e6r, which are decoded into http://localhost:8080/search.json?q=Lek & Lær, where WebRequest.Create gets confused about the &, and thinks it separates parameters, instead of being part of the parameter q.


Solution

  • The solution seems to be to split the query string into each separate component, decoding it and then reencoding it as UTF-8, before joining it all together again.

    string FixQuery(string query){
      return String.Join("&",
        query.Split(new[] {'&'})
          .Select(
            p =>
              String.Join("=",
                p.Split(new[] {'='})
                  .Select(
                    q =>
                      HttpUtility.UrlEncode(
                        HttpUtility.UrlDecode(q),
                        Encoding.UTF8
                      )
                  )
              )
          )
      );
    }
    
    //example usage
    
    var url = "http://localhost:8080/search?"+FixQuery(query);