ASP.NET Core BackgroundService runs several times

Question

I currently have an ASP.NET Core application with the .NET Core SDK v5.0.403.

In this application, I have several BackgroundService defined, one of which being the following:

public class DueDateNotificationService : BackgroundService
{
    private readonly ILogger _logger;
    private readonly IServiceScopeFactory _serviceProvider;

    private readonly int _actionDueDateGenerationDelay;
    private readonly int _startupDelayInSeconds;

    protected static int instances = 0;

    public DueDateNotificationService(ILogger<DueDateNotificationService> logger,
        IServiceScopeFactory serviceProvider,
        IConfiguration configuration)
    {
        _logger = logger;
        _serviceProvider = serviceProvider;

        _actionDueDateGenerationDelay = configuration["BackgroundServices:DueDateNotificationService:DueDateGenerationDelay"].ToInt(240);
        _startupDelayInSeconds = Math.Max(configuration["BackgroundServices:DueDateNotificationService:StartupDelayInSeconds"].ToInt(), 60);
    }

    /// <summary>
    /// Service execution
    /// </summary>
    /// <param name="stoppingToken"></param>
    /// <returns></returns>
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        // Adding basic data to the logging context
        using (LogContext.PushProperty("AppName", "Api-DueDateNotificationService"))
        {
            _logger.LogInformation("DueDateNotificationService Hosted Service is starting.");
            await Task.Delay(TimeSpan.FromSeconds(_startupDelayInSeconds), stoppingToken);

            
                while (!stoppingToken.IsCancellationRequested)
                {


                    try
                    {
                        using (var scope = _serviceProvider.CreateScope())
                        {
                            var service = scope.ServiceProvider.GetRequiredService<IMyService>();
                            _logger.LogInformation($"doing stuff");
                            service.DoStuff();

                        }
                        // waiting until we should check again
                        _logger.LogInformation($"Job finished at {DateTime.UtcNow}. Next job at {DateTime.UtcNow.AddMinutes(_actionDueDateGenerationDelay)}");
                        await Task.Delay(TimeSpan.FromMinutes(_actionDueDateGenerationDelay));
                    }
                    catch (TaskCanceledException)
                    {
                        _logger.LogWarning("The DueDateNotificationService Hosted Service was cancelled, likely because the site was shutdown (possibly due to inactivity or a force update)");
                    }
                    catch (Exception ex)
                    {
                        _logger.LogError(ex, $"Error occurred executing the automated DueDateNotificationService service.");
                    }
                }
            
            _logger.LogInformation("DueDateNotificationService Hosted Service is stopping.");
        
    }
}

My service is added to the DI at one place only, in the startup.cs:

EnableHostedService<DueDateNotificationService>(services, "BackgroundServices:DueDateNotificationService:Enabled");

With EnabledHostedService being:

private IServiceCollection EnableHostedService<T>(IServiceCollection services, string configurationKey) where T : class, IHostedService
        {
            // Background services to run
            if (Configuration[configurationKey].Equals("true", StringComparison.OrdinalIgnoreCase))
            {
                return services.AddHostedService<T>();
            }

            return services;
        }

This is the only place where I register the backgroundService, there is no autofac or custom DI anywhere in the app other than the one provided by default.

My issue is that whenever my app is hosted in an azure app service, ALL the background services are run several times at once.

When I look into my logs,

DueDateNotificationService Hosted Service is starting.

is shown at least twice, and so are all the logs in my while loop (which I removed here to lighten the post).

BUT when I debug it locally, it is run only once, as it should.

When I comment my registration line, it is not started at all (even on Azure).

From what could it come?

Thanks

UPDATE I've even tried with something like:

protected static int instances = 0;
                ....
                Interlocked.Increment(ref instances);
                if (instances > 1)
                {
                    _logger.LogInformation("An instance of DueDateNotificationService is already started.");
                }
                else
                {
                     MyWhileLoop();
                }

It still starts / executes several times, and the following log:

An instance of DueDateNotificationService is already started.

never appears.

Answer 1

azure app service

This is almost certainly due to multiple instances of your app running.

What would be the safest way to have the backgroundservice running only once, beside going through some data persist / check in DB?

An external "lease" is the safest approach. You can build one yourself using CosmosDb / Redis as a backend, or you can use a built-in approach by factoring your background service out of your web app into either an Azure WebJob or an Azure Function. Both of those services use leases automatically for timer-triggered jobs.