Caching in .Net Core with BackgroundService fails: "Adding the specified count to the semaphore would cause it to exceed its maximum count error"

Question

I have implemented a BackgroundService for caching, following exactly the steps described by Microsoft here. I created the default WebApi project, and replaced the fetching of the photos in the Microsoft code with just generating an array of WeatherForecast objects, as that is already available in the sample project. I removed all HttpClient code as well, including DI stuff.

I configure an interval of 1 minute and when I run the code, the CacheWorker.ExecuteAsync method is hit immediately, so all is well. Then, after 1 minute, my breakpoint is hit again only when I hit Continue, the app crashes:

System.Threading.SemaphoreFullException: Adding the specified count to the semaphore would cause it to exceed its maximum count.
   at System.Threading.SemaphoreSlim.Release(Int32 releaseCount)
   at System.Threading.SemaphoreSlim.Release()
   at WebApiForBackgroundService.CacheSignal`1.Release() in D:\Dev\my work\WebApiForBackgroundService\WebApiForBackgroundService\CacheSignal.cs:line 18
   at WebApiForBackgroundService.CacheWorker.ExecuteAsync(CancellationToken stoppingToken) in D:\Dev\my work\WebApiForBackgroundService\WebApiForBackgroundService\CacheWorker.cs:line 61
   at Microsoft.Extensions.Hosting.Internal.Host.TryExecuteBackgroundServiceAsync(BackgroundService backgroundService)
'WebApiForBackgroundService.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\6.0.11\Microsoft.Win32.Registry.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
Microsoft.Extensions.Hosting.Internal.Host: Critical: The HostOptions.BackgroundServiceExceptionBehavior is configured to StopHost. A BackgroundService has thrown an unhandled exception, and the IHost instance is stopping. To avoid this behavior, configure this to Ignore; however the BackgroundService will not be restarted.

The code of my worker service:

using Microsoft.Extensions.Caching.Memory;

namespace WebApiForBackgroundService;

public class CacheWorker : BackgroundService
{
    private static readonly string[] Summaries = new[] { "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching" };
    private readonly CacheSignal<WeatherForecast> _cacheSignal;
    private readonly IMemoryCache _cache;

    public CacheWorker(
        CacheSignal<WeatherForecast> cacheSignal,
        IMemoryCache cache) =>
        (_cacheSignal, _cache) = (cacheSignal, cache);

    public override async Task StartAsync(CancellationToken cancellationToken)
    {
        await _cacheSignal.WaitAsync();
        await base.StartAsync(cancellationToken);
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                WeatherForecast[]? forecasts = Enumerable.Range(1, 5).Select(index => new WeatherForecast
                    {
                        Date = DateTime.Now.AddDays(index),
                        TemperatureC = Random.Shared.Next(-20, 55),
                        Summary = Summaries[Random.Shared.Next(Summaries.Length)]
                    })
                    .ToArray();

                _cache.Set("FORECASTS", forecasts);
            }
            finally
            {
                _cacheSignal.Release();
            }

            try
            {
                await Task.Delay(TimeSpan.FromMinutes(2), stoppingToken);
            }
            catch (OperationCanceledException)
            {
                break;
            }
        }
    }
}

The exception occurs when calling _cacheSignal.Release(), during the 2nd loop, and it's thrown by the CacheSignal class:

namespace WebApiForBackgroundService;

public class CacheSignal<T>
{
    private readonly SemaphoreSlim _semaphore = new(1, 1);
    public async Task WaitAsync() => await _semaphore.WaitAsync();
    public void Release() => _semaphore.Release(); // THROWS EXCEPTION DURING 2ND LOOP
}

And finally my service:

using Microsoft.Extensions.Caching.Memory;

namespace WebApiForBackgroundService;

public sealed class WeatherService : IWeatherService
{
    private readonly IMemoryCache _cache;
    private readonly CacheSignal<WeatherForecast> _cacheSignal;

    public WeatherService(
        IMemoryCache cache,
        CacheSignal<WeatherForecast> cacheSignal) =>
        (_cache, _cacheSignal) = (cache, cacheSignal);

    public async Task<List<WeatherForecast>> GetForecast()
    {
        try
        {
            await _cacheSignal.WaitAsync();

            WeatherForecast[] forecasts =
                (await _cache.GetOrCreateAsync(
                    "FORECASTS", _ =>
                    {
                        return Task.FromResult(Array.Empty<WeatherForecast>());
                    }))!;

            return forecasts.ToList();
        }
        finally
        {
            _cacheSignal.Release();
        }
    }
}

Answer 1

The example seems to be faulty. It seems that idea was to check if nobody uses the cache before it is set for the first time as stated in the article itself:

Important: you need to override BackgroundService.StartAsync and call await _cacheSignal.WaitAsync() in order to prevent a race condition between the starting of the CacheWorker and a call to PhotoService.GetPhotosAsync.

So try changing the ExecuteAsync to the following:

public sealed class CacheWorker : BackgroundService
{
    // ...

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var first = true;
        while (!stoppingToken.IsCancellationRequested)
        {
            _logger.LogInformation("Updating cache.");
            try
            {
                //...
            }
            finally
            {
                if(first)
                {
                    first = false;
                    _cacheSignal.Release();
                }
            }
         }
     }
}

Otherwise you will have endless loop which will try to release semaphore every time while it can have maximum 1 slot (hence the exception).

Links:

• repro with my fix
• docs PR and PR with fix