Search code examples
rustlambda

Why are there lifetime/type mismatch errors when forwarding a reference through a lambda?


I was recently trying to implement a system which allowed for magic type params per rust-magic-function-params, with the addition that the params could be borrowed from the Context object. I built a version of this, with an additional Executor type which can call the Handlers in a generic, type erased way here:

Going through the link, here are the most important parts of the example:

trait Handler<'a, T: 'a>: 'static {
    fn call(&self, value: T) -> Result<(), Error>;
}

impl<'a, T: 'a, F> Handler<'a, T> for F
where
    F: Fn(T) -> Result<(), Error> + 'static,
{
    fn call(&self, value: T) -> Result<(), Error> {
        (self)(value)
    }
}

The Handler trait is responsible for calling the function. It also has a new lifetime parameter 'a, which the value parameter in call() must live for.

trait Param<'a>: Sized + 'a {
    fn from_state(state: &'a HashMap<TypeId, Box<dyn Any>>) -> Result<Self, Error>;
}

#[derive(Clone, Copy, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)]
struct StateVar<'a, T: ?Sized>(&'a T);

impl<'a, T: Any> Param<'a> for StateVar<'a, T> {
    fn from_state(state: &'a HashMap<TypeId, Box<dyn Any>>) -> Result<Self, Error> {
        state
            .get(&TypeId::of::<T>())
            .and_then(|any| any.downcast_ref::<T>().map(StateVar))
            .ok_or_else(|| Error::from("Parameter not present"))
    }
}

The Param<'a> trait is used to create an instance of a variable from the state, which can then be passed to a Handler's parameter (similar to the FromContext trait in the axum example linked above. The StateVar<'a, T> implements the Param<'a> trait for getting references from the state.

struct ErasedHandler(Box<dyn Fn(&HashMap<TypeId, Box<dyn Any>>) -> Result<(), Error>>);

#[derive(Default)]
struct Executor {
    handlers: Vec<ErasedHandler>,
    state: HashMap<TypeId, Box<dyn Any>>,
}

impl Executor {
    #[allow(unused)]
    fn add_handler<'a, H, T>(&mut self, handler: H)
    where
        T: Param<'a> + 'a,
        H: Handler<'a, T>,
    {
        // ...
    }

    fn add_var<T: Any>(&mut self, value: T) {
        self.state.insert(TypeId::of::<T>(), Box::new(value));
    }

    fn run(&self) -> Result<(), Error> {
        for handler in &self.handlers[..] {
            (handler.0)(&self.state)?;
        }
        Ok(())
    }
}

The ErasedHandler holds a type erased handler wrapped in a boxed lambda. The Executor struct holds all registered handlers, as well as the state. The add_var() method adds a StateVar, and the run() method runs all handlers using the contained state.

The implementation of Executor::add_handler is:

impl Executor {
    fn add_handler<'a, H, T>(&mut self, handler: H)
    where
        T: Param<'a> + 'a,
        H: Handler<'a, T>,
    {
        fn make_wrapper<'a_, H_, T_>(
            handler: H_,
        ) -> Box<dyn Fn(&'a_ HashMap<TypeId, Box<dyn Any>>) -> Result<(), Error>>
        where
            T_: Param<'a_> + 'a_,
            H_: Handler<'a_, T_>,
        {
            Box::new(move |state: &'a_ HashMap<TypeId, Box<dyn Any>>| {
                T_::from_state(state).and_then(|param| handler.call(param))
            })
        }

        let wrapper = make_wrapper::<'a, H, T>(handler);
        self.handlers.push(ErasedHandler(wrapper));
    }
}

where it moves the handler into a boxed wrapper lambda, and then adds it to its handlers Vec.

This system can then be called as demonstrated in the main function:

fn main() -> Result<(), Error> {
    fn my_handler(StateVar(value): StateVar<'_, i32>) -> Result<(), Error> {
        println!("Stored state value = {value}");
        Ok(())
    }

    let mut executor = Executor::default();
    executor.add_var::<i32>(42);
    executor.add_handler(my_handler);

    executor.run()?;
    Ok(())
}

Where any number of variables (of unique types) and handlers can be added to run sequentially. When this main function is run, it should ideally print:

Stored state value = 42

However, attempting to run this program results in a compile error:

error: lifetime may not live long enough
  --> src/main.rs:67:42
   |
49 |     fn add_handler<'a, H, T>(&mut self, handler: H)
   |                    -- lifetime `'a` defined here
...
67 |         self.handlers.push(ErasedHandler(wrapper));
   |                                          ^^^^^^^ cast requires that `'a` must outlive `'static`

error[E0308]: mismatched types
  --> src/main.rs:67:42
   |
67 |         self.handlers.push(ErasedHandler(wrapper));
   |                                          ^^^^^^^ one type is more general than the other
   |
   = note: expected trait object `dyn for<'a> Fn(&'a HashMap<TypeId, Box<dyn Any>>) -> Result<(), Box<dyn std::error::Error + Send + Sync>>`
              found trait object `dyn Fn(&HashMap<TypeId, Box<dyn Any>>) -> Result<(), Box<dyn std::error::Error + Send + Sync>>`

For more information about this error, try `rustc --explain E0308`.
error: could not compile `playground` (bin "playground") due to 2 previous errors

These errors are confusing (to me), and I'm not sure how to solve them. Is it possible to write the API in a way that the handler parameters could be borrowed from State, or do they have to be Cloned?

Deleting this line causes the compile error to disappear, but it means that no Handlers will be added to the Executor, and the message won't be printed.


Solution

  • Unfortunately, this cannot be done.

    I'm not so sure what the first error says, but the second error says that ErasedHandle is expected to contain a function that can take &'lifetime HashMap for any lifetime (for<'lifetime> &'lifetime HashMap), but you're actually giving it a closure that can take only &'a HashMap for the specific lifetime 'a defined in add_handler().

    Another way to read it is: what is 'a? we call the handlers inside run() with the lifetime of &self, but this is the lifetime of &self of run(), and it doesn't exist when we call add_handler() before that.

    We need a higher-ranked lifetime, but we unfortunately cannot specify it, I will explain why briefly (because a long explanation will be... long). It is tempting to get rid of 'a and just specify T: for<'a> Param<'a>, but both if we specify H: for<'a> Handler<'a, T> or if we just get rid of 'a in Handler, we hit an uncrossable obstacle: we cannot capture in the type system that we want a closure with HRTB lifetime, that is that takes for<'a> &'a Foo, all we can do is &'a Foo for some specific lifetime 'a.