Search code examples
.netmemory-leaksserilog

Serilog.ILogger.An exception was thrown by a TaskScheduler.Exception of type 'System.OutOfMemoryException' was thrown

I have a .NET 4.8 WindowService which uses Autofac for dependency Injection and uses Serilog for logging.

My Serilog logger is registered as

builder.Register<ILogger>((c, p) =>
{
     return new LoggerConfiguration()
                    .MinimumLevel.Verbose()
                    .WriteTo.Logger(logger => logger
                        .Filter.ByIncludingOnly(e => e.Level == Serilog.Events.LogEventLevel.Verbose || e.Level == Serilog.Events.LogEventLevel.Debug)
                        .WriteTo.Async(asyncLog => asyncLog.File(ApplicationConstants.ApplicationLogPath + "\\Trace\\Trace-.txt", Serilog.Events.LogEventLevel.Verbose, ApplicationConstants.ApplicationLogTemplate, rollingInterval: RollingInterval.Day, rollOnFileSizeLimit: true, fileSizeLimitBytes: 10000000, shared: true)))
                    .WriteTo.Logger(logger => logger
                        .Filter.ByExcluding(e => e.Level == Serilog.Events.LogEventLevel.Verbose && e.Level == Serilog.Events.LogEventLevel.Debug)
                        .WriteTo.Map("LogContext", "Log",
                        (LogContext, wt) => wt.Async(asyncLog => asyncLog.File(ApplicationConstants.ApplicationLogPath + $".\\Logs\\{LogContext}-.txt", Serilog.Events.LogEventLevel.Information, ApplicationConstants.ApplicationLogTemplate, rollingInterval: RollingInterval.Day, rollOnFileSizeLimit: true, fileSizeLimitBytes: 10000000, shared: true))))
                    .CreateLogger();
});

I use factory pattern to resolve multiple classes that uses the same interface. Their dependencies are registered as follow

 builder.RegisterType<HandleA>().As<IHandler>().Keyed<IHandler>("HandleA");
 builder.RegisterType<HandleB>().As<IHandler>().Keyed<IHandler>("HandleB");

The service continuously process data and depending on type it then resolve this dependency on runtime like this

case "A":
    return ContainerManager.Container.ResolveKeyed<IHandler>("HandleA");
case "B":
    return ContainerManager.Container.ResolveKeyed<IHandler>("HandleB");

where ContainerManager.Container is a static IContainer variable that contains the container when all the dependencies are built on service start.

Now the implementation of the class of the interface IHandler is

public class HandleA: IHandler
{
     private ILogger _fileLogger;
     public HandleA(ILogger fileLogger)
     {
            _fileLogger = fileLogger;
     }
}

At this moment of resolution, the code throws error

An exception was thrown while activating Service.HandleA
 -> ?:Serilog.ILogger.An exception was thrown by a TaskScheduler.Exception of type 'System.OutOfMemoryException' was thrown.
at Autofac.Core.Resolving.Middleware.ActivatorErrorHandlingMiddleware.Execute(ResolveRequestContext context, Action`1 next)     
at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.<BuildPipeline>b__1(ResolveRequestContext ctxt)     
at Autofac.Core.Pipeline.ResolvePipeline.Invoke(ResolveRequestContext ctxt)     
at Autofac.Core.Resolving.Middleware.RegistrationPipelineInvokeMiddleware.Execute(ResolveRequestContext context, Action`1 next)     
at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.<BuildPipeline>b__1(ResolveRequestContext ctxt)     
at Autofac.Core.Resolving.Middleware.SharingMiddleware.Execute(ResolveRequestContext context, Action`1 next)    
 at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.<BuildPipeline>b__1(ResolveRequestContext ctxt)    
 at Autofac.Core.Resolving.Middleware.ScopeSelectionMiddleware.Execute(ResolveRequestContext context, Action`1 next)     
 at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.<BuildPipeline>b__1(ResolveRequestContext ctxt)    
 at Autofac.Core.Resolving.Middleware.CircularDependencyDetectorMiddleware.Execute(ResolveRequestContext context, Action`1 next)    
 at Autofac.Core.Resolving.Pipeline.ResolvePipelineBuilder.<>c__DisplayClass14_0.<BuildPipeline>b__1(ResolveRequestContext ctxt)   
 at Autofac.Core.Pipeline.ResolvePipeline.Invoke(ResolveRequestContext ctxt)   
 at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, ResolveRequest request)  
 at Autofac.Core.Resolving.ResolveOperation.ExecuteOperation(ResolveRequest request)   
 at Autofac.Core.Lifetime.LifetimeScope.ResolveComponent(ResolveRequest request)    
 at Autofac.Core.Container.ResolveComponent(ResolveRequest request)   
 at Autofac.ResolutionExtensions.TryResolveService(IComponentContext context, Service service, IEnumerable`1 parameters, Object& instance)    
 at Autofac.ResolutionExtensions.ResolveService(IComponentContext context, Service service, IEnumerable`1 parameters)     
 at Autofac.ResolutionExtensions.ResolveKeyed[TService](IComponentContext context, Object serviceKey, IEnumerable`1 parameters)

NOTE: This does not happen every time when dependency is resolved by occasionally and only reproduces on a server (690 out of 8333 times) and not locally.

Because of its sporadic nature, I cannot continuously monitor the Task Manager to check memory usage.

Server has 32 cores and 64-128GB RAM and has multiple services and API's hosted on it so the problem is definitely on my end.

Solution

  • One thing which is very suspicious is the way how you are registering the logger. By default Instance Per Dependency strategy is used by Autofac, so you will create a logger per dependency which is not how Serilog is supposed to be used AFAIK.

    Consider either building the logger once at startup and registering it as instance:

    var logger = new LoggerConfiguration()
    .MinimumLevel.Verbose()
    .WriteTo.Logger(logger => logger
        .Filter.ByIncludingOnly(e => e.Level == Serilog.Events.LogEventLevel.Verbose || e.Level == Serilog.Events.LogEventLevel.Debug)
        .WriteTo.Async(asyncLog => asyncLog.File(ApplicationConstants.ApplicationLogPath + "\\Trace\\Trace-.txt", Serilog.Events.LogEventLevel.Verbose, ApplicationConstants.ApplicationLogTemplate, rollingInterval: RollingInterval.Day, rollOnFileSizeLimit: true, fileSizeLimitBytes: 10000000, shared: true)))
    .WriteTo.Logger(logger => logger
        .Filter.ByExcluding(e => e.Level == Serilog.Events.LogEventLevel.Verbose && e.Level == Serilog.Events.LogEventLevel.Debug)
        .WriteTo.Map("LogContext", "Log",
            (LogContext, wt) => wt.Async(asyncLog => asyncLog.File(ApplicationConstants.ApplicationLogPath + $".\\Logs\\{LogContext}-.txt", Serilog.Events.LogEventLevel.Information, ApplicationConstants.ApplicationLogTemplate, rollingInterval: RollingInterval.Day, rollOnFileSizeLimit: true, fileSizeLimitBytes: 10000000, shared: true))))
    .CreateLogger();

// ...

builder.RegisterInstance<ILogger>(logger);

    Or

    builder.Register<ILogger>((c, p) =>
{
   // your current code
   // ...
}
.SingleInstance();

    Also maybe worth looking into using something like Serilog.Extensions.Autofac.DependencyInjection.