Net Core Application Insights - How to set the connection string after application startup?

Question

I use Application Insights across several .NET Core 7 projects for sending log telemetry data to Azure. Across most of my projects, I use Azure Key Vault to eliminate storing connection strings in the app.

For my IOT project, requirements are different and am hitting a chicken and egg scenario.

Firstly using Azure Key Vault is not viable for this IOT Project (its not going to be hosted in Azure and is designed to startup as an unattended device). The approach I'm using for security is using TPM Authentication for connection to Azure IOT Hub, which once it gets the device twin properties back, it fetches other secrets and connection strings needed to startup other services. In essence, no secrets/connection strings are ever stored in the IOT project, it has to reach Azure IOT Hub to get this information.

From the above, I naturally can't provide App Insights a connection string during the app startup, I need to be able to pass it the connection string once it's connected with Azure IOT Hub, which is further down the line from the startup. I'm obviously not bothered about sending telemetry to Azure until the device has connected with the Hub, I just log events to a local text file for this preliminary startup process.

Is it possible to somehow initialize App Insights during startup but then give it the connection string it wants a bit later on? or is there some other way to initiaize App Insights without doing it during startup, and just make use of the service when I ready to do so?

The MS Docs generally say to provide the connection string when intilaizing the service, but I'm not convinced doing this without the connection string is going to allow it to magically start working further down the line.

I know its possible to create a telemetry client elsewhere in the app without registering it at startup, my thoughts are this may not be the best solution if sending lots of log data assuming it has to create a fresh connection each time I send telemetry.

Below is a code sample of what I usually use in Program.cs (which obviously needs some adjustments to suit the above scenario:

// Configure the Azure Key Vault. We action this straight away in order we can use it to get
// the instrumentation key needed for app insights, which we need to setup before configuring
builder.Configuration.AddAzureKeyVault(
        new Uri($"https://{builder.Configuration["AzureKeyVaultConfig:KeyVaultName"]}.vault.azure.net/"),
        new DefaultAzureCredential());

// Create the connection string for App Insights.
var appInsightConnectionString = "InstrumentationKey="
    + builder.Configuration["ApplicationInsights:InstrumentationKey"]
    + ";IngestionEndpoint="
    + builder.Configuration["ApplicationInsights:IngestionEndpoint"];

// ----- Configure Application Insights Telemetry -----
Microsoft.ApplicationInsights.AspNetCore.Extensions.ApplicationInsightsServiceOptions aiOptions = new();
aiOptions.ConnectionString = appInsightConnectionString; // See appInsightConnectionString at the top of the page.
aiOptions.EnableQuickPulseMetricStream = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableQuickPulseMetricStream");
aiOptions.EnableEventCounterCollectionModule = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableEventCounterCollectionModule");
aiOptions.EnableAppServicesHeartbeatTelemetryModule = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableAppServicesHeartbeatTelemetryModule");
aiOptions.EnableAzureInstanceMetadataTelemetryModule = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableAzureInstanceMetadataTelemetryModule");
aiOptions.EnableDependencyTrackingTelemetryModule = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableDependencyTrackingTelemetryModule");
aiOptions.EnableEventCounterCollectionModule = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableEventCounterCollectionModule");
aiOptions.EnableAdaptiveSampling = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableAdaptiveSampling");
aiOptions.EnableHeartbeat = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableHeartbeat");
aiOptions.AddAutoCollectedMetricExtractor = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("AddAutoCollectedMetricExtractor");
// Web App Options
aiOptions.EnableRequestTrackingTelemetryModule = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableRequestTrackingTelemetryModule");
aiOptions.EnableAuthenticationTrackingJavaScript = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableAuthenticationTrackingJavaScript");
aiOptions.EnableDiagnosticsTelemetryModule = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableDiagnosticsTelemetryModule");
aiOptions.EnableActiveTelemetryConfigurationSetup = builder.Configuration.GetSection("ApplicationInsights").GetValue<bool>("EnableActiveTelemetryConfigurationSetup");
// Build the serive with the options from above.
//builder.Services.AddLogging(loggingBuilder => loggingBuilder.AddFilter<Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider>("", LogLevel.Trace));
builder.Services.AddApplicationInsightsTelemetry(aiOptions);

Answer 1

My solution so far involves creating a static instance of the telemetry client which is initialized only once we have a valid connection string.

The connection string can be set at any time we chose (in my case, some time after the app has started up). As per the notes in my original question, in my environment I only need to start sending telemetry to Azure once I've fetched the connection string securely from the Azure IOT Hub. All logging up to this point would use the baked in ILogger instance alone, of which I simply log to the console and local log files.

When I started looking for solutions, I was getting some mixed messages about the creation of the Telemetry Client and whether this should be done each time I write a new log message, or create a singleton style instance and make reference to the same one for the lifetime of the app. Looking at the AppInsights SDK itself, it looks to be the service is usually created as a singleton, also going by the fact that the client instance is usually created during the app startup for use with dependancy injection.

Conclusions from some other posts around the web:

• Telemetry Client is deemed thread safe, but also in this solution, once we've initialized it, we're not making any changes.
• Another post had said they suffered a memory leak by creating a new instance each time a new message was logged, so creating a static instance seems like the safer solution.

With credit to the following post found in the 'summary' notes noted below, we create a simple class for initializing the Telemetry Client:

/// <summary>
/// See https://briancaos.wordpress.com/2020/05/07/c-azure-telemetryclient-will-leak-memory-if-not-implemented-as-a-singleton/
/// </summary>
public class AppInsightsTelemetryFactory
{
    private static TelemetryClient? TelemetryClient { get; set; }

    public static TelemetryClient GetTelemetryClient()
    {
        if (TelemetryClient == null)
        {
            if (!String.IsNullOrEmpty(IotDeviceSettingsRepo.AzureAppInsightsConnectionString))
            {
                var telemetryConfiguration = new TelemetryConfiguration
                {
                    ConnectionString = IotDeviceSettingsRepo.AzureAppInsightsConnectionString
                };
                TelemetryClient = new TelemetryClient(telemetryConfiguration);
            }
            else 
            { 
                return null!; 
            }
        }

        return TelemetryClient;
    }
}

I also created a static getter/setter for the connection string which I've put in a seperate class, but nothing stops you placing in the same class as above.

/// <summary>
/// String is updated once the device twin properties 
/// have been retrieved from the Azure IOT Hub.
/// </summary>
private static string? _AzureAppInsightsConnectionString = String.Empty; // Default
private readonly static object _AzureAppInsightsConnectionStringLock = new();

public static string? AzureAppInsightsConnectionString
{
    get { lock (_AzureAppInsightsConnectionStringLock) return _AzureAppInsightsConnectionString; }
    set { lock (_AzureAppInsightsConnectionStringLock) _AzureAppInsightsConnectionString = value; }
}

The connection string is then set with a value once I've retrieved it from the hub:

IotDeviceSettingsRepo.AzureAppInsightsConnectionString = "myConnectionString";

Then finally, anywhere within the app where we want to send telemetry, we simply fetch the telemetry client instance (which in turn checka whether or not the connection string is available). If the GetTelemetryClient() method returns null then the telemetryClient has not yet been initialized and therefore the telemetryClient?.TrackTrace will not run.However once the connection string ha ben set, the next time we log an event, the GetTelemetryClient() method will return a valid instance and the events are then sent to Azure:

var telemetryClient = AppInsightsTelemetryFactory.GetTelemetryClient();

// Log Information
telemetryClient?.TrackTrace(JsonSerializer.Serialize(le), SeverityLevel.Information);

// Log Warning
telemetryClient?.TrackTrace(JsonSerializer.Serialize(le), SeverityLevel.Warning);

// Log Error
telemetryClient?.TrackTrace(JsonSerializer.Serialize(le), SeverityLevel.Error);

Side note: I'm serializing the log event payload for use with structured logging, which is why the TrackTrace has "JsonSerializer.Serialize(le)" just saying as to avoid any confusion in my example and not relavant to the solution in hand :)