Sleuth instrumentation of Spring cloud stream messages is lost when using zipkin + kafka

Question

I have the following setup :

• several Spring Cloud Edgware services with most notably the spring-cloud-starter-zipkin and spring-cloud-stream-binder-kafka dependencies.
• a Zipkin server running the openzipkin/zipkin:2.8 docker image
• I see traces from the service going into the "zipkin" topic, zipkin picks those traces and I can look at them in the zipkin ui.
• everything seems OK at first sight, especially for rest call between my services, but I just saw that it doesn't work for my own messages that I use for asynchronous communications between my services (I use spring cloud stream for this).
• I can see the log messages from TraceChannelInterceptor indicating that the instrumentation is taking place
• but the trace message headers are not present in the message when I inspect the data in kafka.

There used to be a StreamEnvironmentPostProcessor that did the job of adding the trace headers to the kafka bindings when I included the spring-cloud-sleuth-stream dependendy in the past. But the doc clearly states now :

Note: spring-cloud-sleuth-stream is deprecated and incompatible with these destinations

What should I do to make this work properly now ? Add the headers to the bindings configuration myself ? Or is there something I'm missing ?

Answer 1

That was a bug in Spring Cloud Sleuth in Edgware. The Stream Kafka Binder in Edgware required explicit passing of headers that should get propagated. The side effect of adding sleuth-stream on the classpath was exactly that feature. By fixing the https://github.com/spring-cloud/spring-cloud-sleuth/issues/1005 issue we're adding back the missing feature to core. This is not ported to Finchley since Stream Kafka Binder in Finchley passes all headers by default.

The workaround for Edgware is to pass a list of headers in the following manner:

spring:
  cloud:
    stream:
      kafka:
        binder:
          headers:
            - spanId
            - spanSampled
            - spanProcessId
            - spanParentSpanId
            - spanTraceId
            - spanName
            - messageSent