I have an ASP.NET Core Web API I'm developing locally. It uses two different authentication schemes, both JWT bearer schemes - one is for "user" tokens (issued by my own Identity provider) and the other is for "service" tokens which are issued by my Azure tenant. I'll add some code below to show the setup of these schemes, but I'm not doing anything unusual.
Just recently I'm finding that my protected endpoints are suddenly extremely slow to respond in my local development environment - in the order of 60-second response times. I have found this is only true for endpoints that require service tokens. For example, an endpoint that accepts user or service tokens and is called with a user token responds fine when the service token requirement is removed.
Also, if I put a breakpoint in some middleware that runs before authentication then this breakpoint is hit fast, then a huge delay before the breakpoint in the controller itself is hit.
So I've concluded that something inside the JWT bearer authentication middleware is causing the app to hang. But I have no idea what this might be. I've tried using Fiddler to inspect the internal network traffic being made by my app (netsh winhttp set proxy 127.0.0.1:8888
) but nothing there that's particularly long-running. Below is the trace. One thing that stands out here is the absence of any request to the Azure tenant well-known discovery doc. The first two requests are to my own discovery doc for the user tokens. Everything else seems to be App Insights-related.
# Result Protocol Host URL Body Caching Content-Type Process Comments Custom
12 200 HTTPS my.identity.local:44119 /.well-known/openid-configuration 2,104 application/json; charset=UTF-8 identity.app:800
13 200 HTTPS my.identity.local:44119 /.well-known/openid-configuration/jwks 463 application/json; charset=UTF-8 identity.app:800
17 200 HTTP www.msftconnecttest.com /connecttest.txt 22 max-age=30, must-revalidate text/plain svchost:3132
18 502 HTTP ipv6.msftconnecttest.com /connecttest.txt 512 no-cache, must-revalidate text/html; charset=UTF-8 svchost:3132
24 200 HTTPS fe2cr.update.microsoft.com /v6/ClientWebService/client.asmx 2,155 private text/xml; charset=utf-8 svchost:8068
34 502 HTTP ipv6.msftconnecttest.com /connecttest.txt 512 no-cache, must-revalidate text/html; charset=UTF-8 svchost:3132
35 200 HTTP www.msftconnecttest.com /connecttest.txt 22 max-age=30, must-revalidate text/plain svchost:3132
47 200 HTTPS fe2cr.update.microsoft.com /v6/ClientWebService/client.asmx 2,149 private text/xml; charset=utf-8 svchost:8068
I've also tried tapping into the JWT bearer events. The only two being hit for the Service scheme are MessageReceived
(hit very fast) and AuthenticationFailed
(after the long delay and as expected because the token being used isn't a service token) so no clues there.
Hard to believe such a huge delay is CPU-bound, but I really don't know what else I can do to investigate. Have any of you encountered this before? Any tips would be amazing because it's so frustrating having such an insanely slow development environment. Like I say, this has only started in the last couple of days and I've barely touched the API codebase. Is it possible I'm being subjected to some sort of Azure rate limiting? I have a paid Azure account. As promised, here's the extension method I use to set up this authentication scheme:
private static void AddServiceTokenAuthenticationIfRequired(this AuthenticationBuilder authBuilder, IServiceCollection services, AuthenticationSettings settings)
{
if (!settings.SupportServiceTokens)
{
return;
}
services.AddSingleton<CustomServiceAuthenticationEvents>();
var tenantId = settings.TenantId ?? throw new Exception("Token tenant not configured");
var authority = $"https://sts.windows.net/{tenantId}";
var audience = settings.DefaultTokenAudience ?? throw new Exception("Token audience not configured");
authBuilder.AddJwtBearer(AuthenticationSchemes.Service, options =>
{
options.Audience = audience;
options.Authority = new Uri(authority).AbsoluteUri;
options.TokenValidationParameters.NameClaimType = JwtClaimTypes.Subject;
options.TokenValidationParameters.ValidateIssuer = true;
options.TokenValidationParameters.ValidIssuers = [$"{authority}/"];
options.TokenValidationParameters.ValidateAudience = true;
options.TokenValidationParameters.ValidateIssuerSigningKey = true;
options.TokenValidationParameters.ValidateLifetime = true;
options.TokenValidationParameters.LogValidationExceptions = true;
options.EventsType = typeof(CustomServiceAuthenticationEvents);
options.MapInboundClaims = false;
});
}
OK, after 24 hours of banging my head against a wall I discovered that the massive latency between my dev laptop and Microsoft was being caused by the Cloudflare WARP app I'd installed to supposedly speed up my internet. I can't explain it but this app has caused me other problems in the past with network flakiness so I'm just uninstalling it. Disabling/uninstalling fixes the issue which, by the way, applied to all communication between my machine and Azure resources so not just authentication supporting resources.