Azure Application Insight Custom Telemetry Failures

Question

I am logging the custom telemetry events using Telemetry Client in azure function. It logs the custom event almost every time certain event is triggered but it fails occasionally say once in a thousandth time. I can ascertain that no exception is thrown and event is processed successfully without any error. Here is an example of how I am logging the custom telemetry.

        try
            {
                _telemetryClient.TrackEvent("MyCustomEventName",
                    new KeyValuePair<string, string>("CustomEvent",
                        JsonSerializer.Serialize(customEventObject)));
            }
            catch (Exception e)
            {
                _logger.LogError(e,"Failed to log CustomEventObject");
            }

I am logging the exception if telemetry client fails and logger shows no exception. Is there any reason Telemetry Client could fail to dispatch the event? If yes, how to diagnose and handle it?

I am on .net core 3.1 with Azure Function Runtime 3.1.3.0

Answer 1

You can use the following two ways of sampling to diagnose and handle the failure of dispatching the event:

• Adaptive sampling automatically adjusts the volume of telemetry sent from the SDK in your ASP.NET/ASP.NET Core app, and from Azure Functions. This is the default sampling when you use the ASP.NET or ASP.NET Core SDK. Adaptive sampling is currently only available for ASP.NET server-side telemetry, and for Azure Functions.

• Fixed-rate sampling reduces the volume of telemetry sent from both your ASP.NET or ASP.NET Core or Java server and from your users' browsers. You set the rate. The client and server will synchronize their sampling so that, in Search, you can navigate between related page views and requests.

There are two AdaptiveSamplingTelemetryProcessor nodes added by default, and one includes the Event type in sampling, while the other excludes the Event type from sampling.

<TelemetryProcessors>
    <Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor, Microsoft.AI.ServerTelemetryChannel">
        <MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
        <ExcludedTypes>Event</ExcludedTypes>
    </Add>
    <Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor, Microsoft.AI.ServerTelemetryChannel">
        <MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
        <IncludedTypes>Event</IncludedTypes>
    </Add>
</TelemetryProcessors>

You can refer to Types of Sampling, How can I track telemetry that's not automatically collected?, Telemetry sampling without affecting the errors/failures and open GitHub issue at How do you correctly get TelemetryClient dependency injected in ASP.NET Core?