Search code examples
spring-cloud-sleuth

Custom transport for Sleuth traces

My project is using Spring Cloud Sleuth to participate in recording distributed spans. Communication with other systems is done over Dapr, which is not supported by Spring Cloud Sleuth. Applications communicatie with each other through a sidecar (offered by Dapr); communication between the app and the sidecar is HTTP or gRPC.

Dapr expects distributed traces to be propaged according to the W3C standard. Using the W3C standard is possible in Sleuth by setting the following in application.yml:

spring:
  sleuth:
    propagation:
      type: w3c

However, this only works for transports that are managed by Spring. Since Dapr is not tied to Spring or any other framework, they roll their own clients for HTTP or gRPC communication with the sidecar. As a result, the Sleuth instrumentation does not work.

I can't find any hints in the documentation how I would write my own instrumentation.

So my question is: how would I propagate those W3C-compatible metadata using a non-Spring managed connection?

Solution

  • Spring Cloud Sleuth does two things form a very high perspective:

    1. Gives you an API over tracing libraries (abstracts away the tracing libs)
    2. Instruments Spring projects or libraries that are supported by Spring

    Because of this, if you have a library that is not supported by Spring, you can check if the underlying tracing library supports it or not. By default the tracing lib is Brave but it does not seem it supports Dapr.

    In the case of neither your tracing library nor Sleuth does not instrument the project of your choice, you need to write your own instrumentation (you can also open an issue for Brave to support it).

    You can use Sleuth and/or Brave to write instrumentation, these parts of the docs can help:

    On top of these you need to figure out how to transfer tracing data over the wire. In most cases this happens using headers (HTTP, Kafka, AMQP/RabbitMQ, etc.) on the client side, you need to add the tracing data to your message so that it is transferred over the wire and on the server side you need to fetch this data and reuse it if any (if there is none, you need to create a Span from scratch).

    You can take a look how other ~similar transports are instrumented (in Sleuth or Brave) and do something similar, e.g.: HTTP, Kafka, AMQP/RabbitMQ, gRPC, Dubbo, etc.