Tracing Kafka bus in distributed app using Jaeger

Question

I have distribute application that consists of several Go services. Some of those use Kafka as data bus. I was able trace down calls between services using opentracing with Jaeger. I have problem plotting Kafka spans on graph, them appear as gaps.

Here is what i was able to do. Initial spans been created by gRPC middleware. Producer side:

...    
kafkaMsg := kafka.Message{Key: []byte(key), Value: msgBytes}
headers:=make(map[string]string)
if span := opentracing.SpanFromContext(ctx); span != nil {
    opentracing.GlobalTracer().Inject(
        span.Context(),
        opentracing.TextMap,
        opentracing.TextMapCarrier(headers))
}
for headerKey, headerValue:=range headers{
    msg.Headers = append(msg.Headers, kafka.Header{
        Key: headerKey,
        Value: []byte(headerValue),
    })
}
// Write message to Kafka
...

Consumer side:

...
// read headers from Kafka message
headers := make(map[string]string)
for _, header := range kafkaMessage.Headers{
    headers[header.Key]=string(header.Value)
}
spanContext, _ := opentracing.GlobalTracer().Extract(opentracing.TextMap, opentracing.TextMapCarrier(headers))
span := opentracing.StartSpan(
    "Consumer",
    opentracing.FollowsFrom(spanContext))
defer span.Finish()
...

How should i modify this to plot span on graph when message was in Kafka?

Answer 1

So, answering my own question. Jaeger does not support cross system spans. Every sub-system is responsible for its own span in the whole system. For reference, check this answer https://github.com/opentracing/specification/issues/143