C# URL encoding (net481)

Question

I try to build a search query for a movie website, and me being a European we have a bit of weird characters.

This is my starting code:

private static string GetSearchUrl(string name)
{
    var uriBuilder = new UriBuilder("https://www.themoviedb.org/search");
    var query = HttpUtility.ParseQueryString(uriBuilder.Query);
    query.Add("query", name);
    uriBuilder.Query = query.ToString();
    var searchUrl = uriBuilder.ToString();
    return searchUrl;
}

these are my test cases:

const string Source = "Zur Hölle mit den Paukern";
const string Source = "Astérix & Obélix";

When I create a unit test library in net6.0 everything is very fine, the results are:

https://www.themoviedb.org:443/search?query=Zur+H%c3%b6lle+mit+den+Paukern
https://www.themoviedb.org:443/search?query=Ast%c3%a9rix+%26+Ob%c3%a9lix

However, running the very same function in my target framework net481 I get

https://www.themoviedb.org:443/search?query=Zur+H%u00f6lle+mit+den+Paukern
https://www.themoviedb.org:443/search?query=Ast%u00e9rix+%26+Ob%u00e9lix

The ö character is encoded as %c3%b6 in net6.0 but as %u00f6 in net481. The é is encoded as %c3%a9 in net6.0 but as %u00e9 in net481. And that doesn't work as a call URL (at least not on themoviedb.org)

I tried encoding as converting from UTF-8 to Windows-1252 first:

var win1252 = Encoding.GetEncoding(1252).GetString(Encoding.UTF8.GetBytes(name));
query.Add("query", win1252);

But it made no difference.

I also toyed with Uri.EscapeDataString() in various places but it usually made it worse.

In net6.0 the actual type behind the query is

System.Collections.Specialized.NameValueCollection
{System.Web.HttpUtility.HttpQSCollection}

In net481 it is

System.Collections.Specialized.NameValueCollection
{System.Web.HttpValueCollection}

but these are internal implementations returned by the HttpUtility.ParseQueryString() call

Answer 1

With the help of Heretic Monkey I decided to code my own version of the QueryBuilder which works for my purposes in net6.0 and net481:

public sealed class HttpGetQueryBuilder
{
    private readonly Dictionary<string, string> _queryParameters;

    public HttpGetQueryBuilder()
    {
        _queryParameters = new Dictionary<string, string>();
    }

    public static HttpGetQueryBuilder ParseQueryString(string query)
    {
        var instance = new HttpGetQueryBuilder();
        var keyValues = HttpUtility.ParseQueryString(query);
        foreach (var key in keyValues.AllKeys)
        {
            instance.Add(key, keyValues[key]);
        }
        return instance;
    }

    public string this[string key]
    {
        get => _queryParameters[key];
        set => _queryParameters[key] = value;
    }

    public void Add(string key, string value)
        => _queryParameters.Add(key, value);

    public override string ToString()
    {
        var queryBuilder = new StringBuilder();
        var keyIndex = 0;
        foreach (var keyValue in _queryParameters)
        {
            queryBuilder.Append(Uri.EscapeDataString(keyValue.Key));
            queryBuilder.Append('=');
            queryBuilder.Append(Uri.EscapeDataString(keyValue.Value));
            keyIndex++;
            if (keyIndex < _queryParameters.Count)
            {
                queryBuilder.Append('&');
            }
        }
        return queryBuilder.ToString();
    }
}