How can I transfer a stacktrace betwen Scala ExecutionContexts?

Question

I have written a small utility wrapper around the Scala ExecutionContext to enable transfer of MDC contexts across Future instances. It works as expected but an undesired side-effect is that we now don't seem to get stack traces that go across Futures. How can I ensure that the stack trace is propagated along with the MDC?

Here's my code for reference:

import org.slf4j.MDC

import scala.concurrent.ExecutionContext
import scala.jdk.CollectionConverters._


object MdcOps {

  implicit class ExecutionContextExt(value: ExecutionContext) {
    def withMdc: ExecutionContext = new MdcExecutionContext(value)
  }

  def withMdc[A](mdc: Map[String, Any], replace: Boolean)(doIt: => A): A = {
    val currentMdcContext = getMdc
    val newMdc = if(replace) mdc else mdc ++ currentMdcContext
    try { setMdc(newMdc); doIt }
    finally { setMdc(currentMdcContext) }
  }

  def setMdc(mdc: Map[String, Any]): Unit = {
    if(mdc.isEmpty) {
      MDC.clear()
    } else
      MDC.setContextMap(mdc.view.mapValues(_.toString).toMap.asJava)
  }

  def getMdc: Map[String, String] = Option(MDC.getCopyOfContextMap).map(_.asScala.toMap).getOrElse(Map.empty)
}

class MdcExecutionContext(underlying: ExecutionContext, context: Map[String, String] = Map.empty) extends ExecutionContext {
  override def prepare(): ExecutionContext = new MdcExecutionContext(underlying, MdcOps.getMdc)
  override def execute(runnable: Runnable): Unit = underlying.execute { () =>
    MdcOps.withMdc(context, replace = true)(runnable.run())
  }
  override def reportFailure(t: Throwable): Unit = underlying.reportFailure(t)
}

Answer 1

In general it is hard. ZIO implemented it - see: https://www.slideshare.net/jdegoes/error-management-future-vs-zio from slide 53 to see the result - but creation of stacks, and passing it around causes performance penalty x2.4.

Since you cannot change API to e.g. generate stack trace in compile times using macros (using e.g. sourcode library), on each .map/.flatMap you would have to create a stack trace, remove irrelevant frames from it, maybe combine it with frames from previous context and set it in MDC. Putting technical details aside - this is hard to get right and heavy weight, more like a material for a whole library than simple utility. If it could be done cheaply this would be a build in.

If you are very interested in it, either pick ZIO or analyze how it implements it - from what I see it required a lot of effort and building this functionality into the library, and even then it causes performance penalty. I can only imagine that penalty by using JVM stack traces and passing them around would be even greater.