Search code examples
kotlinkotlin-coroutinesmicrometerspring-micrometer

How do I prevent Micrometer observations from being stopped when a suspending function is called?


I have a Spring application written in Kotlin that uses the Micrometer observation library (1.10.3). The application executes a suspend function that is annotated with Micrometer's @Observed annotation. The function calls a Kotlin coroutine await function within it.

I have noticed while inspecting the logs of observation events that the observation started by the @Observered annotation is stopped (and any open scopes for the observation are closed) right before it calls the Kotlin coroutine await function. This causes any observation events created after the await function to be created in the wrong observation (or in no observation at all). How do I prevent this from happening?

As a side note, I've also noticed that within the ObservedAspect class's code, there is handling for methods that have a CompletionStage return type. However, modifying the return type of my @Observed function changes nothing, and the return type is always evaluated as just java.util.Object. I've noticed that this only happens if the function being @Observed has the suspend keyword, and the return type evaluates correctly if there is no suspend.


Solution

  • I haven't found the perfect solution yet, but my workaround has been to do the following:

    1. Update my spring-aop dependency to the latest version which fixes the handling of Kotlin suspend functions in the interceptor.(https://github.com/spring-projects/spring-framework/issues/22462)
    2. Create a custom annotation, because the current ObservedAspect does not handle waiting for the Mono returned by the join point (an effect of step 1) before closing the observation scope. This was an issue because observation scopes would just open and close immediately after the Mono was returned. By the time the Mono is subscribed to and the function being proxied is actually executed, there would be no more observation to emit events to.
    3. Wrap the functions we have annotated with @Observed in a mono block using the coroutine context created by ObservationRegistry#asContextElement(). This allows the observation registry to retain the ThreadLocal values of the current observation scope before and after any coroutines get executed within the proxied method. This fixes our issue where it would no longer be possible to retrieve the current observation from the observation registry when the thread is suspended since the thread would be different once the method resumes.

    Hopefully, in the future, the @Observed annotation and ObservedAspect interceptors would be able to handle observations scoping reactive publishers out of the box.

    It would also be cleaner if there was a way to specify the coroutine context to use for the proxied method from the interceptor, instead of needing to wrap every method to be proxied in a mono block and specifying the custom coroutine context there.