401 Unauthorized while using Identity authentication and authorization in ASP.NET Core 3.1

Question

I am getting 401 Unauthorized while trying to access endpoints like clients like Insomnia, curl or regular get methods like httpContext.GetAsync(url);, but can access the endpoints through regular browsers like Edge or Chrome.

I am new to web dev and I am using ASP.NET Core 3.1 to develop a simple web app that would read an Excel file and save the contents to a DB. I am storing the CreatedBy field using username. I should not have an explicit login page to get the username and password, instead I have to use the Windows username (PC-Name\LoginName).

The app would be used only under a closed network where all the Windows users are registered with the network so I need not check anything at all. So the logged in user id would be 'PC-Name\LoginName'

The problem I am facing is that, the endpoints work just fine when I access them thru a browser (atleast it returns PC-Name\LoginName without any problems), but I get 401 Unauthorized when I try to access them thru clients like Insomnia, Curl or even when I make internal API calls like _httpContext.GetAsync(url);. I also tried using the exact same headers that my browser used in Insomia, but I still got the same result. I am mostly just missing something really simple, but I can't find out what.

My Startup.cs file

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.HttpsPolicy;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Authentication.Negotiate;

namespace TestingWebAPI
{
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddAuthentication(NegotiateDefaults.AuthenticationScheme).AddNegotiate();
            services.AddAuthorization(options => { options.FallbackPolicy = options.DefaultPolicy; });
            services.AddControllers();
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }

            app.UseHttpsRedirection();

            app.UseRouting();

            app.UseAuthentication();
            app.UseAuthorization();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers();
            });
        }
    }
}

My Controller action

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;

namespace TestingWebAPI.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class ValuesController : ControllerBase
    {
        [HttpGet]
        [Route("{action}")]
        public string GetCurrentUser()
        {
            return HttpContext.User.Identity.Name;
        }
    }
}

My launchSettings.json

{
  "$schema": "http://json.schemastore.org/launchsettings.json",
  "iisSettings": {
    "windowsAuthentication": true,
    "anonymousAuthentication": true,
    "iisExpress": {
      "applicationUrl": "http://localhost:60085",
      "sslPort": 44323
    }
  },
  "profiles": {
    "IIS Express": {
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "weatherforecast",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      }
    },
    "TestingWebAPI": {
      "commandName": "Project",
      "launchBrowser": true,
      "launchUrl": "weatherforecast",
      "applicationUrl": "https://localhost:5001;http://localhost:5000",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      }
    }
  }
}

Browser Request Headers:

GET /api/Values/GetCurrentUser HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate, br, zsdch
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Cookie: csrftoken=gy4oKWvUo9WRpaaWMgp6DiFuTZnfUsTb
DNT: 1
Host: localhost:44323
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.183
sec-ch-ua: "Not/A)Brand";v="99", "Microsoft Edge";v="115", "Chromium";v="115"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"

Answer 1

A 401 response is the normal first step when using Windows Authentication. That 401 response includes a WWW-Authenticate header specifying the supported authentication methods. For Windows Authentication, it would be Negotiate and/or NTLM.

If the website is trusted (in the Trusted Sites in Internet Options) then Chrome and Edge will automatically retry the request with an Authorization header that has the credentials of the currently-logged-on user.

When you use a tool like curl, that automatic second request doesn't happen unless you tell it to, like with:

curl --negotiate -u : https://example.com

or

curl --ntlm -u : https://example.com

In C#, using HttpClient, you can tell it to send the current user's credentials by declaring it like this:

new HttpClient(new HttpClientHandler() { UseDefaultCredentials = true })