Search code examples
spring-bootibm-mqspring-jmsspring-cloud-sleuthzipkin

Spring cloud Sleuth starts a new trace instead of continuing spans in a single trace


I have 4 spring-boot applications (A, B, C and D). The lifecycle of a transaction is as follows :

  1. Application A is a kafka streams application and it ultimately produces to a topic which is consumed by Application B.
  2. Application B then consumes from the topic using @KafkaListener, does some processing and then produces to IBMMQ queue using spring's jmsTemplate.
  3. Application C which is a @JMSListener consumes from the above queue and produces to another queue using spring's JMSTemplate.
  4. Application D which is again a @JmsListener consumes from the above queue and then produces to a kafka topic, which the again consumed by Application A

Now for a single transaction I would expect a single trace across all four application, but instead I get

  1. One Trace starting from application A to application B (where it produces to IBM MQ)
  2. One trace starting from Application C and ending at Application A

I would have uploaded the pictures to show the zipkin spans, but for some reason I am not able to do so.

All the above applications are Spring boot applications and they utilize spring-cloud-sleuth for producing transactions traces. I am relying on spring boot's autoconfiguration and these are the properties that I have set in all the applications:

  zipkin:
   enabled: ${ZIPKIN_ENABLED:false}
   sender:
    type: kafka
   baseUrl: ${ZIPKIN_URL:http://localhost:9411}
   service:
    name: ${spring.application.name}
 sleuth:
  messaging:
   kafka:
    enabled: true
   jms:
    enabled: true

I am not able to understand what's exactly happening here. Why the spans are scattered across 2 traces and not one?

I am using spring-boot 2.3.3 and spring-cloud-dependencies Hoxton.SR8.


Solution

  • So it was application B which was not passing the header along. Turns out that the queue uri had a property targetClient which was set to 1. The uri is something like

    queue:///DESTINATION_QUEUE?targetClient=1
    

    Now I am not an IBM MQ expert by far, but the documentation states that setting this property to 1 means that Messages do not contain an MQRFH2 header. I toggled it to 0 and voila, all spans fall into place.