.NET HttpClient do not persist authentication between reqeusts to IIS when using NTLM Negotiate

IIS site configured to use windows authentication with default options. The client is written in C# and uses single HttpClient instance to perform requests. Requests success, but every request triggers 401 Challenge:

Traffic captured with Wireshark. We performed loud test, and noticed, that with anonymous authentication client performs 5000 reqeusts per second, but with windows authentication - 800. So, looks like wireshark does not impact to authentication, performance slowdown indicates, that 401 Challenge also occurs without wireshark.

Wirehshark log: https://drive.google.com/file/d/1vDNZMjiKPDisFLq6ZDhASQZJJKuN2cpj/view?usp=sharing

Code of client is here:

var httpClientHandler = new HttpClientHandler();
httpClientHandler.UseDefaultCredentials = true;
var httpClient = new HttpClient(httpClientHandler);
while (working)
{
    var response = await httpClient.GetAsync(textBoxAddress.Text + "/api/v1/cards/" + cardId);
    var content = await response.Content.ReadAsStringAsync();
}

IIS site settings:

How to make HttpClient to persist authentication between requests, to prevent waste negotiate handshakes on every request?

UPD: Code of client: https://github.com/PFight/httpclientauthtest Steps to reproduce:

Create folder with simple file index.html, create application 'testsite' in IIS for this folder. Enable anonymous authentication.
Run client (https://github.com/PFight/httpclientauthtest/blob/main/TestDv5/bin/Debug/TestDv5.exe), press start button - see count of requests per second. Press stop.
Disable anonymous atuhentication, enable windows authentication.
Press start button in client, see count of reqeusts per second.

On my computer I see ~1000 requests per second on anonymous, and ~180 on windows. Wireshark shows 401 challenges on every request for windows authentication. keep-alive header enabled in IIS.

IIS version: 10.0.18362.1 (windows 10)

Version of System.Net.Http.dll loaded to process: 4.8.3752.0

Solution

Firstly I tried to save the Authorization header for re-use it with every new request.

using System.Net.Http.Headers;

Requester requester = new Requester();
await requester.MakeRequest("http://localhost/test.txt");
await Task.Delay(100);
await requester.MakeRequest("http://localhost/test.txt");

class Requester
{
    private readonly HttpClientHandler _httpClientHandler;
    private readonly HttpClient _httpClient;
    private AuthenticationHeaderValue _auth = null;

    public Requester()
    {
        _httpClientHandler = new HttpClientHandler();
        _httpClientHandler.UseDefaultCredentials = true;
        _httpClient = new HttpClient(_httpClientHandler);
        _httpClient.DefaultRequestHeaders.Add("User-Agent", Guid.NewGuid().ToString("D"));
    }

    public async Task<string> MakeRequest(string url)
    {
        HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Get, url);
        message.Headers.Authorization = _auth;

        HttpResponseMessage resp = await _httpClient.SendAsync(message);
        _auth = resp.RequestMessage?.Headers?.Authorization;
        resp.EnsureSuccessStatusCode();
        string responseText = await resp.Content.ReadAsStringAsync();
        return responseText;
    }
}

But it didn't work. Every time there was http code 401 asking for authentication despite of Authorization header.

The IIS logs is listed below.

2021-12-23 15:07:47 ::1 GET /test.txt - 80 - ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 401 2 5 127
2021-12-23 15:07:47 ::1 GET /test.txt - 80 MicrosoftAccount\[email protected] ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 200 0 0 4
2021-12-23 15:07:47 ::1 GET /test.txt - 80 - ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 401 1 2148074248 0
2021-12-23 15:07:47 ::1 GET /test.txt - 80 MicrosoftAccount\[email protected] ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 200 0 0 0

IIS's Failed Requests Tracing reports the following when receiving re-used Authentication Header:

Property	Value
ModuleName	WindowsAuthenticationModule
Notification	AUTHENTICATE_REQUEST
HttpStatus	401
HttpReason	Unauthorized
HttpSubStatus	1
ErrorCode	The token supplied to the function is invalid (0x80090308)

I've made a research and I can say that this is not possible without alive connection.

Every time the connection is closed there will be new handshake.

According this and this answers NTLM authenticates a connection, so you need to keep your connection open.

NTLM over http is using HTTP persistent connection or http keep-alive.

A single connection is created and then kept open for the rest of the session.

If using the same authenticated connection, it is not necessary to send the authentication headers anymore.

This is also the reason why NTLM doesn't work with certain proxy servers that don't support keep-alive connections.

UPDATE:

I found the key point using your example.

First: You must enable keep-alive at your IIS

Second: You must set authPersistSingleRequest flag to false. Setting this flag to True specifies that authentication persists only for a single request on a connection. IIS resets the authentication at the end of each request, and forces re-authentication on the next request of the session. The default value is False.

Third: You can force the HttpClient to send keep-alive headers:

httpClient.DefaultRequestHeaders.Add("Connection", "keep-alive");
httpClient.DefaultRequestHeaders.Add("Keep-Alive", "600");

Using this three key points I have achieved only one NTLM handshake during connection lifetime.

Also it's important which version of .NET \ .NET Framework do you use. Because HttpClient hides different realizations dependent on framework version.

Framework	Realization of HttpClient
.Net Framework	Wrapper around WebRequest
.Net Core < 2.1	Native handlers (WinHttpHandler / CurlHandler)
.Net Core >= 2.1	SocketsHttpHandler

I tried it on .NET 6 and it works great, but it didn't work on .Net Framework as I can see, so here is the question: which platform do you use?

UPDATE 2:

Found the solution for .Net Framework.

CredentialCache myCache = new CredentialCache();
WebRequestHandler handler = new WebRequestHandler()
{
    UseDefaultCredentials = true,
    AllowAutoRedirect = true,
    UnsafeAuthenticatedConnectionSharing = true,
    Credentials = myCache,
};
var httpClient = new HttpClient(handler);

httpClient.DefaultRequestHeaders.Add("Connection", "keep-alive");
httpClient.DefaultRequestHeaders.Add("Keep-Alive", "600");

var from = DateTime.Now;
var countPerSecond = 0;
working = true;
while (working)
{
    var response = await httpClient.GetAsync(textBoxAddress.Text);
    var content = await response.Content.ReadAsStringAsync();
    countPerSecond++;
    if ((DateTime.Now - from).TotalSeconds >= 1)
    {
        this.labelRPS.Text = countPerSecond.ToString();
        countPerSecond = 0;
        from = DateTime.Now;
    }

    Application.DoEvents();
}

The key point is to use WebRequestHandler with UnsafeAuthenticatedConnectionSharing enabled option and use credential cache.

If this property is set to true, the connection used to retrieve the response remains open after the authentication has been performed. In this case, other requests that have this property set to true may use the connection without re-authenticating. In other words, if a connection has been authenticated for user A, user B may reuse A's connection; user B's request is fulfilled based on the credentials of user A.

Caution Because it is possible for an application to use the connection without being authenticated, you need to be sure that there is no administrative vulnerability in your system when setting this property to true. If your application sends requests for multiple users (impersonates multiple user accounts) and relies on authentication to protect resources, do not set this property to true unless you use connection groups as described below.

Big thanks to this article for solution.