Propagate top-level span ID's in OpenTelemetry

Question

I'm trying to get OpenTelemetry tracing working with FastAPI and Requests. Currently, my setup looks like this:

import requests
from opentelemetry.baggage.propagation import W3CBaggagePropagator
from opentelemetry.propagators.composite import CompositePropagator
from fastapi import FastAPI
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.requests import RequestsInstrumentor
from opentelemetry.propagate import set_global_textmap
from opentelemetry.propagators.b3 import B3MultiFormat
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator

set_global_textmap(CompositePropagator([B3MultiFormat(), TraceContextTextMapPropagator(), W3CBaggagePropagator()]))

app = FastAPI()

FastAPIInstrumentor.instrument_app(app)
RequestsInstrumentor().instrument()

@app.get("/")
async def get_things():
    r = requests.get("http://localhost:8081")

    return {
        "Hello": "world",
        "result": r.json()
    }

The / endpoint just does a GET to another service that looks basically like this one, just with some middleware to log the incoming headers.

If I send a request like this (httpie format),

http :8000 'x-b3-traceid: f8c83f4b5806299983da51de66d9a242' 'x-b3-spanid: ba24f165998dfd8f' 'x-b3-sampled: 1'

I expect that the downstream service, i.e. the one being requested by requests.get("http://localhost:8081"), to receive headers that look something like

{
  "x-b3-traceid": "f8c83f4b5806299983da51de66d9a242",
  "x-b3-spanid": "xxxxxxx",  # some generated value from the upstream service
  "x-b3-parentspanid": "ba24f165998dfd8f", 
  "x-b3-sampled": "1"
}

But what I'm getting is basically exactly what I sent to the upstream service:

{
  "x-b3-traceid": "f8c83f4b5806299983da51de66d9a242",
  "x-b3-spanid": "ba24f165998dfd8f",
  "x-b3-sampled": "1"
}

I must be missing something obvious, but can't seem to figure out exactly what.

Sending a W3C traceparent header results in the same exact situation (just with traceparent in the headers that are received downstream). Any pointers would be appreciated.

EDIT - I'm not using any exporters, as in our environment, Istio is configured to export the traces. So we just care about the HTTP traces for now.

Answer 1

The B3MultiFormat propagator doesn't consider the parent span id field while serialising the context into HTTP headers since X-B3-ParentSpanId is an optional header https://github.com/openzipkin/b3-propagation#multiple-headers. You can expect the X-B3-TraceId and X-B3-SpanId to be always present but not the remaining ones.

Edit:

Are you setting the concrete tracer provider? It doesn't look like from the shared snippet but I don't know if you are actual application code. It's all no-op if you do not set the sdk tracer provider i.e no recording spans are created in FastAPI service. Please do the following.

...
from opentelemetry.trace import set_tracer_provider
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.resources import Resource

set_tracer_provider(TracerProvider(
    resource=Resource.create({"serice.name": "my-service"})
))

...

Another edit:

OpenTelemetry does not store the parent span ID in the context https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/api.md#spancontext. The context propagation client libraries from OTEL are limited to serialise and pass on this info only. I don't think you can have the parentSpanId propagated.