Search code examples
azureazure-web-app-service.net-5azure-linux

Azure web app on Linux stoped working suddenly


I have a web application on .NET Core 5 deployed on the Azure web app (Linux). Deployment to the web app goes via Azure DevOps and the last deployment took place at the end of April and no one touched the web app in any shape or form.

However, suddenly the application stoped and was responding with the default Azure page 'Your app service is up and running. According to the AppInsights, the application was not under any heavy load only a few people on the web CPU and RAM around 20%. The 'Always on' setting is turned on.

According to the logs below, I assume that the machine running my web app was restarted for some reason (I am guessing Azure maintenance) I see that it was unable to find files necessary to start thus it went with the default Azure page. However, I do not understand why as the files were there, moreover, as you can see at 9:52 we manually restarted the web app through Azure portal and it fixed the issue.

I am looking for advice on preventing this behaviour as this is the second time it happened, the first time was like 5 months ago. Thanks


/appsvctmp/volatile/logs/runtime/bf30e8c59ba178a56a30e0d094f31d1ca4dfed5fef11f3a5f521ec418a4f26a0.log 
2021-06-10T09:52:45.113197498Z: [INFO]  ASP .NETCore Version: 5.0.3
2021-06-10T09:52:45.113202598Z: [INFO]  Note: Any data outside '/home' is not persisted
2021-06-10T09:52:45.136558888Z: [INFO]  Running oryx create-script -appPath /home/site/wwwroot -output /opt/startup/startup.sh -defaultAppFilePath /defaulthome/hostingstart/hostingstart.dll     -bindPort 8080 -userStartupCommand '' 
2021-06-10T09:52:45.143249243Z: [INFO]  Cound not find build manifest file at '/home/site/wwwroot/oryx-manifest.toml'
2021-06-10T09:52:45.143290643Z: [INFO]  Could not find operation ID in manifest. Generating an operation id...
2021-06-10T09:52:45.143644646Z: [INFO]  Build Operation ID: d16c01f9-bcff-40dc-9efb-5ece15c2dbdc
2021-06-10T09:52:46.219921700Z: [INFO]  Writing output script to '/opt/startup/startup.sh'
2021-06-10T09:52:46.609737971Z: [INFO]  Trying to find the startup DLL name...
2021-06-10T09:52:46.609773771Z: [INFO]  Found the startup D name: MyApp.dll
2021-06-10T09:52:46.609780971Z: [INFO]  Running the command: dotnet "MyApp.dll"


/appsvctmp/volatile/logs/runtime/9e3c16f1b56384c8bac17db8e768f5f5099fe35a4991801a6e83e4f337d68c9c.log 
2021-06-10T09:43:08.196901399Z: [INFO]  Build Operation ID: 51765fea-d04c-49e3-b4e4-9a4bbbc68436
2021-06-10T09:43:09.231514603Z: [INFO]  An error occurred while trying to look for '.runtimeconfig.json' files under '/home/site/wwwroot'.
2021-06-10T09:43:09.685030062Z: [INFO]  WARNING: Unable to find the startup DLL name. Could not find any files with extension '.runtimeconfig.json'
2021-06-10T09:43:10.119271456Z: [INFO]  Writing output script to '/opt/startup/startup.sh'
2021-06-10T09:43:10.294162644Z: [INFO]  Trying to find the startup DLL name...
2021-06-10T09:43:10.294223844Z: [INFO]  Running the default app using command: dotnet "/defaulthome/hostingstart/hostingstart.dll"
2021-06-10T09:43:10.460518558Z: [INFO]  Hosting environment: Production
2021-06-10T09:43:10.460618459Z: [INFO]  Content root path: /defaulthome/hostingstart/
2021-06-10T09:43:10.460630159Z: [INFO]  Now listening on: http://[::]:8080
2021-06-10T09:43:10.460634759Z: [INFO]  Application started. Press Ctrl+C to shut down.


/appsvctmp/volatile/logs/runtime/45188b981fba42d5a512175877e15093cac84c9a73b0296a782fb5b64882be69.log 
2021-06-10T09:38:11.131655396Z: [INFO]  Build Operation ID: db220961-7c36-4051-b3f5-668b96e819a9
2021-06-10T09:38:12.454815722Z: [INFO]  An error occurred while trying to look for '.runtimeconfig.json' files under '/home/site/wwwroot'.
2021-06-10T09:38:12.636611612Z: [INFO]  WARNING: Unable to find the startup DLL name. Could not find any files with extension '.runtimeconfig.json'
2021-06-10T09:38:13.064061230Z: [INFO]  Writing output script to '/opt/startup/startup.sh'
2021-06-10T09:38:13.647112171Z: [INFO]  Trying to find the startup DLL name...
2021-06-10T09:38:13.647167671Z: [INFO]  Running the default app using command: dotnet "/defaulthome/hostingstart/hostingstart.dll"
2021-06-10T09:38:14.165655564Z: [INFO]  Hosting environment: Production
2021-06-10T09:38:14.165687265Z: [INFO]  Content root path: /defaulthome/hostingstart/
2021-06-10T09:38:14.165693365Z: [INFO]  Now listening on: http://[::]:8080
2021-06-10T09:38:14.165697865Z: [INFO]  Application started. Press Ctrl+C to shut down.

Filesystem under the wwwroot folder

AutoMapper.dll
Cronos.dll
Humanizer.dll
MyApp
MyApp.BackOffice.Rcl.Views.dll
MyApp.BackOffice.Rcl.Views.pdb
MyApp.BackOffice.Rcl.dll
MyApp.BackOffice.Rcl.pdb
MyApp.Domain.dll
MyApp.Domain.pdb
MyApp.Domain.xml
MyApp.Infrastructure.dll
MyApp.Infrastructure.pdb
MyApp.deps.json
MyApp.dll
MyApp.pdb
MyApp.runtimeconfig.json
MyApp.xml
JWT.dll
Joonasw.AspNetCore.SecurityHeaders.dll
MetadataExtractor.dll
Microsoft.AI.DependencyCollector.dll
Microsoft.AI.EventCounterCollector.dll
Microsoft.AI.PerfCounterCollector.dll
Microsoft.AI.ServerTelemetryChannel.dll
Microsoft.AI.WindowsServer.dll
Microsoft.ApplicationInsights.AspNetCore.dll
Microsoft.ApplicationInsights.dll
Microsoft.AspNetCore.Authentication.JwtBearer.dll
Microsoft.AspNetCore.AzureAppServices.HostingStartup.dll
Microsoft.AspNetCore.AzureAppServicesIntegration.dll
Microsoft.AspNetCore.Identity.EntityFrameworkCore.dll
Microsoft.AspNetCore.Mvc.Razor.Extensions.dll
Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation.dll
Microsoft.AspNetCore.Razor.Language.dll
Microsoft.Azure.KeyVault.WebKey.dll
Serilog.Formatting.Compact.dll
Serilog.Settings.Configuration.dll
Serilog.Sinks.ApplicationInsights.dll
Serilog.Sinks.Async.dll
Serilog.Sinks.Console.dll
Serilog.Sinks.Debug.dll
Serilog.Sinks.File.dll
Serilog.Sinks.PeriodicBatching.dll
Serilog.Sinks.Seq.dll
Serilog.Sinks.Slack.dll
Serilog.dll
StarkbankEcdsa.dll
Swashbuckle.AspNetCore.Annotations.dll
Swashbuckle.AspNetCore.Swagger.dll
Swashbuckle.AspNetCore.SwaggerGen.dll
Swashbuckle.AspNetCore.SwaggerUI.dll
System.Composition.AttributedModel.dll
System.Composition.Convention.dll
System.Composition.Hosting.dll
System.Composition.Runtime.dll
System.Composition.TypedParts.dll
System.Configuration.ConfigurationManager.dll
System.Diagnostics.DiagnosticSource.dll
System.Diagnostics.PerformanceCounter.dll
System.IdentityModel.Tokens.Jwt.dll
System.Runtime.Caching.dll
System.Security.Cryptography.ProtectedData.dll
Twilio.dll
XmpCore.dll
appsettings.json
appsettings.production.json
appsettings.staging.json
cs
de
dotnet-aspnet-codegenerator-design.dll
es
fr
it
ja
ko
pl
pt-BR
refs
ru
runtimes

Solution

  • Apparently, the maintenance you have mentioned is a Storage Failover (File Server Reboot of Volume Path Change), when this happens your the platform switches your application to a secondary storage.

    Not finding the needed files to startup is something that happens sometimes with .NET Core apps due to the build process not finding the correct DLL to run while the app is still starting up.

    In the meantime, specifying the DLL to be run in the startup command would be a good approach to avoid this issue happening again.

    In the Portal -> Configuration -> General Settings -> Startup Command:

    dotnet example.dll