Search code examples
azure-cosmosdbilogger

CreateContainerIfNotExistsAsync hangs, stalls, never returns


I have a working Blazor app that reads and writes to CosmosDb. Trying to add some new functionality, which works fine locally, but bombs after deploying to Azure. I've traced the problem to somewhere in my CosmosDbService. I want to add ILogger into CosmosDbService and further trace down my problem. However, while testing the DI, my code stalls at CreateDatabaseIfNotExistsAsync(databasename). No errors in the prior code. No exceptions, no timeouts, nothing just a silent fail. The Azure is not reporting anything. If I roll everything back to original state, it works. Obviously it is there somewhere, but I am not seeing it. So my questions are 1) Why does my app now stall/hang? 2) Am I injecting ILogger correctly into my CosmosDbService? Here is the relevant code.

Startup.cs
....
            services.AddSingleton<ICosmosDbService>((s) =>
            {
                var logger = s.GetRequiredService<ILogger<CosmosDbService>>();
                return InitializeCosmosClientInstanceAsync(Configuration.GetSection("CosmosDb"), logger).GetAwaiter().GetResult();
            });
'''
        private static async Task<CosmosDbService> InitializeCosmosClientInstanceAsync(IConfigurationSection configurationSection, ILogger logger)
        {
            string databaseName = configurationSection.GetSection("DatabaseName").Value;
            string containerRatings = configurationSection.GetSection("ContainerRatings").Value;
            string containerUsers = configurationSection.GetSection("ContainerUsers").Value;
            string containerLocations = configurationSection.GetSection("ContainerLocations").Value;

            string account = configurationSection.GetSection("Account").Value;
            string key = configurationSection.GetSection("Key").Value;
            CosmosClientBuilder clientBuilder = new CosmosClientBuilder(account, key);
            CosmosClient client = clientBuilder
                                .WithConnectionModeDirect()
                                .WithSerializerOptions(new CosmosSerializationOptions { PropertyNamingPolicy = CosmosPropertyNamingPolicy.Default })
                                .Build();
            //CosmosDbService cosmosDbService = new CosmosDbService(client, databaseName, containerRatings, containerUsers, containerLocations);
            CosmosDbService cosmosDbService = new CosmosDbService()
            {
                ContainerRatings = client.GetContainer(databaseName, containerRatings),
                ContainerRateLocationSmall = client.GetContainer(databaseName, containerLocations),
                ContainerRateUser = client.GetContainer(databaseName, containerUsers),
                CosmosClient = client,
                DatabaseName = databaseName
                , _logger = logger
            };

            DatabaseResponse database = await client.CreateDatabaseIfNotExistsAsync(databaseName);
            await database.Database.CreateContainerIfNotExistsAsync(containerRatings, "/What");
            await database.Database.CreateContainerIfNotExistsAsync(containerUsers, "/UserName");
            await database.Database.CreateContainerIfNotExistsAsync(containerLocations, "/id");
            return cosmosDbService;
        }
...
CosmosDBService.cs
...
    public class CosmosDbService : ICosmosDbService
    {
        public CosmosClient CosmosClient { get; set; }
        public string DatabaseName { get; set; }
        public Container ContainerRatings { get; set; }
        public Container ContainerRateUser { get; set; }
        public Container ContainerRateLocationSmall { get; set; }

        public ILogger _logger { get; set; }
        //public CosmosDbService() { }
...

Thanks in advance.

--UPDATE 2020-12-06 ---

Thank you Quango. Your questions and suggestions stimulated a new line of thinking. I'm giving you credit for the answer.

  1. I noticed that the new line in Startup.cs was causing the service to lazy load much later in the actual program execution:
                var logger = s.GetRequiredService<ILogger<CosmosDbService>>();

When I returned this line to its original syntax, it initialized during Startup successfully.

            services.AddSingleton<ICosmosDbService>(InitializeCosmosClientInstanceAsync(Configuration.GetSection("CosmosDb")).GetAwaiter().GetResult());
  1. Getting ILogger injected into my CosmosDBService. Doing re-RTFM on Logging (https://learn.microsoft.com/en-us/aspnet/core/fundamentals/logging/?view=aspnetcore-5.0) it seemed I needed to use the Factory instead. So I changed the instantiation to this:
            LoggerFactory loggerFactory = new LoggerFactory();
            //CosmosDbService cosmosDbService = new CosmosDbService(client, databaseName, containerR8ings, containerUsers, containerLocations);
            CosmosDbService cosmosDbService = new CosmosDbService()
            {
                ContainerR8ings = dbClient.GetContainer(databaseName, containerR8ings),
                ContainerR8User = dbClient.GetContainer(databaseName, containerUsers),
                ContainerR8LocationSmall = dbClient.GetContainer(databaseName, containerLocations),
                _logger = loggerFactory.CreateLogger("CosmosDbService")               
            };

And correspondingly to the CosmosDBService by removing the explicit constructor and using the implicit one. Then I was able to add Logger to the service.

  1. Finally this enabled me to track the original bug I was hunting.

Thanks again for the assistance.


Solution

  • As you've seen you cannot asynchronously initialize a service in the service registration process. The solution I use in these cases is not to create the service directly, but return a factory or provider service which has an async Task<T> CreateAsync method to do this. You register this provider as the service, then inject this provider and call CreateAsync in the code. Here is an example:

    using Microsoft.Azure.Cosmos;
    using Microsoft.Azure.Cosmos.Fluent;
    using Microsoft.Extensions.Configuration;
    using Microsoft.Extensions.Logging;
    using System;
    using System.Threading.Tasks;
    
    namespace TestWebsite.Services
    {
        public class CosmosDbProvider
        {
            /// <summary>
            /// Ctor: values provided by DI
            /// </summary>
            /// <param name="configuration"></param>
            /// <param name="logger"></param>
            public CosmosDbProvider(IConfiguration configuration, ILogger logger)
            {
                config = configuration;
                log = logger;
            }
    
            IConfiguration config;
            ILogger log;
    
            ICosmosDbService cachedService;
    
            public async Task<ICosmosDbService> CreateAsync()
            {
                // use a cached instance for all requests (similar to Lazy<T> but async)
                if (cachedService == null)
                {
                    cachedService = await InitializeCosmosClientInstanceAsync();
                }
                return cachedService;
            }
    
            private async Task<CosmosDbService> InitializeCosmosClientInstanceAsync()
            {
                try
                {
                    log.LogInformation("InitializeCosmosClientInstanceAsync started");
    
                    // I'd recommend GetValue<T> for reading single values over GetSection
                    string databaseName = config.GetValue<string>("DatabaseName");
                    string containerRatings = config.GetValue<string>("ContainerRatings");
                    string containerUsers = config.GetValue<string>("ContainerUsers");
                    string containerLocations = config.GetValue<string>("ContainerLocations");
    
                    string account = config.GetValue<string>("Account");
                    string key = config.GetValue<string>("Key");
    
                    log.LogDebug($"databaseName: {databaseName}");
                    log.LogDebug($"containerRatings: {containerRatings}");
                    log.LogDebug($"containerUsers: {containerUsers}");
                    log.LogDebug($"containerLocations: {containerLocations}");
                    // might not want to log sensitive info..
                    // log.LogDebug($"account: {account}");
                    // log.LogDebug($"key: {key}");
    
                    log.LogDebug("Creating CosmosClientBuilder");
                    CosmosClientBuilder clientBuilder = new CosmosClientBuilder(account, key);
                    
                    log.LogDebug("Creating CosmosClient");
                    CosmosClient client = clientBuilder
                                        .WithConnectionModeDirect()
                                        .WithSerializerOptions(new CosmosSerializationOptions { PropertyNamingPolicy = CosmosPropertyNamingPolicy.Default })
                                        .Build();
    
                    log.LogDebug("Creating CosmosDbService");
                    CosmosDbService cosmosDbService = new CosmosDbService()
                    {
                        ContainerRatings = client.GetContainer(databaseName, containerRatings),
                        ContainerRateLocationSmall = client.GetContainer(databaseName, containerLocations),
                        ContainerRateUser = client.GetContainer(databaseName, containerUsers),
                        CosmosClient = client,
                        DatabaseName = databaseName,
                        _logger = log
                    };
    
                    log.LogDebug($"CreateDatabase for '{databaseName}'");
                    DatabaseResponse database = await client.CreateDatabaseIfNotExistsAsync(databaseName);
                    
                    log.LogDebug($"CreateContainer for '{containerRatings}'");
                    await database.Database.CreateContainerIfNotExistsAsync(containerRatings, "/What");
                    
                    log.LogDebug($"CreateContainer for '{containerUsers}'");
                    await database.Database.CreateContainerIfNotExistsAsync(containerUsers, "/UserName");
    
                    log.LogDebug($"CreateContainer for '{containerLocations}'");
                    await database.Database.CreateContainerIfNotExistsAsync(containerLocations, "/id");
    
                    log.LogInformation("InitializeCosmosClientInstanceAsync completed");
                    return cosmosDbService;
    
                }
                catch (Exception e)
                {
                    log.LogError(e, "Failed when initializing CosmosDB");
                    throw;
                }
    
            }
        }
    }
    

    I changed your config code to use GetValue<T> - I think it's simpler, and added a bunch of logging statements. Mostly these are LogDebug so in production you'll need to change to debug-level (or you can change to LogInformation if you wish) to help diagnose the issue.

    In Startup.cs we add:

    services.AddSingleton<CosmosDbProvider>();
    

    To get the CosmosDB instance you inject the provider using DI. In controllers, Razor and Blazor pages you can then make async calls: e.g. a sample Blazor page Cosmos.razor

    @page "/cosmos"
    @inject TestWebsite.Services.CosmosDbProvider cosmosProvider;
    
    <h3>Cosmos</h3>
    
    @if(_db !=null)
    {
        <p>Your database is called @_db.DatabaseName</p>
    }
    
    @code {
    
        TestWebsite.Services.ICosmosDbService _db;
    
        protected override async Task OnInitializedAsync()
        {
            // create/get CosmosDbService
            _db = await cosmosProvider.CreateAsync();
        }
    }