Why am I getting 403 on GetAsync from api.stackexchange?

Question

I am writing a demo app for myself to learn and suddenly I started getting 403 when doing api.stackexchange.com requests and I fail at EnsureSuccessStatusCode(). I'm doing them to populate my database with some tags data. (injected client just sets AutomaticDecompression to GZip)

public async Task<List<TagsImport>> ImportStackOverflowTagsAsync()
{
    var tagImportsList = new List<TagsImport>();
    var httpClient = _httpClient.GetClient();

    httpClient.BaseAddress = new Uri("https://api.stackexchange.com/");
    httpClient.DefaultRequestHeaders.Accept.Clear();

    for (var i = 1; i <= 10; i++)
    {
        var response = await httpClient.GetAsync($"2.3/tags?page={i}&pagesize=100&order=desc&sort=popular&site=stackoverflow");
        response.EnsureSuccessStatusCode();

        var responseBody = await response.Content.ReadAsStringAsync();

        var tags = JsonSerializer.Deserialize<TagsImport>(responseBody);
        tagImportsList.Add(tags);
    }

    return tagImportsList;
}

Tried manually testing the request and then testing previous versions of the app, the test in the app just returns "System.Net.Http.HttpRequestException : Response status code does not indicate success: 403 (Forbidden)." while manually writing "https://api.stackexchange.com/2.3/tags?page=1&pagesize=100&order=desc&sort=popular&site=stackoverflow" in browser returns the data properly.

Am I missing something to make it work?

Answer 1

This error is caused by User-Agent header. Basically what you need to do is that add user agent header into your httpClient and it would behave like a browser getting the data without issues.

copy & paste this:

 httpClient.DefaultRequestHeaders.UserAgent.TryParseAdd("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36");