I've been diving deep into Azure Application Insights and have thoroughly reviewed this. Despite this, I'm still unclear about a key concept.
Azure Application Insights seems to break down a trace into requests and dependencies.
What is the rationale behind this separation?
In other tracing systems, I'm used to seeing traces as a continuous sequence of spans, so I'm confused about why Azure has chosen this approach. Is there a way to view the actual OTLP (OpenTelemetry Protocol) traces within Azure Application Insights (using their queries that is)?
I was expecting to find a comprehensive trace in the traces table, but instead, I only see this segmented information (request + depenendency)
Additionally, I've tried the following two configurations in my .NET app:
// Attempt #1
builder.Services.AddApplicationInsightsTelemetry(builder.Configuration["ApplicationInsights:ConnectionString"]);
// Attempt #2
builder.Services.AddOpenTelemetry()
.WithTracing(builder2 => builder2
.AddSource("YourAppName")
.SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("YourServiceName"))
.AddAspNetCoreInstrumentation()
.AddHttpClientInstrumentation()
.AddAzureMonitorTraceExporter(o =>
{
o.ConnectionString = builder.Configuration["ApplicationInsights:ConnectionString"];
}));
var app = builder.Build();
Both yielded the same results so far, but I found some documentation suggesting that the second approach should give me OTLP traces. I did see OTLP traces (under both attempts), but only for exceptions.
Questions:
Any insights or examples would be greatly appreciated. Thanks
These are good questions. Let me try to answer some of them / provide workarounds.
Part of the answer - it is historical. Application Insights is 10+ year old product and first appeared where the most common pattern was monolith application. Since then Kubernetes, microservices and as a result Distributed Tracing became essential part of modern applications. Application Insights is already fully aligned with Open Telemetry on SDK/app side but still stores data in existing schema for distributed tracing to work across applications instrumented with Open Telemetry and instrumented with previous technologies.
While Application Insights splits spans into two different tables, End-to-end transaction details
visualizes the entire trace (and pulls data not only from these two tables but from any other Application Insights resource which participated in a particular trace). For instance, if there are N applications and they ingest telemetry in their own Application Insights
resources - the below experience will automatically pull data from all resources which participated in a particular trace. There is also a link which will transition to Logs with a query returning everything for a particular trace (across all resources and tables).
There is ongoing work which will introduce a new OTelSpans
table. And will provide backward and forward compatibility (i.e., it will be possible to query data using new and old schemas). Though the work is in progress, at this point cannot say when it will land.
Meanwhile it is possible to merge these two tables into one using union
KQL keyword.
union requests, dependencies
| count