Search code examples
rustlifetime

Generic HKT bounds with lifetimes


When creating interpreters for dynamically-typed DSLs, writing builtin functions quickly becomes tedious due to the need to type-check all arguments. To solve this, I've created a trait that does this for me.

The DowncastArg casts from the enum &Value to it's variants. The ExtractArgs trait converts a list of arguments &[Value] to a tuple of arguments, each downcast to it's proper type.

I have it almost working but I'm running into a lifetime issue (playground):

#[derive(Debug, Eq, PartialEq)]
enum Value {
    Int(i64),
    String(String),
}

trait DowncastArg<'a>: Sized + 'a {
    fn downcast_arg(value: &'a Value) -> Option<Self>;
}

impl DowncastArg<'_> for i64 {
    fn downcast_arg(value: &Value) -> Option<Self> {
        if let Value::Int(variant) = value {
            Some(*variant)
        } else {
            None
        }
    }
}

impl<'a> DowncastArg<'a> for &'a str {
    fn downcast_arg(value: &'a Value) -> Option<Self> {
        if let Value::String(variant) = value {
            Some(variant)
        } else {
            None
        }
    }
}

trait ExtractArgs<'a>: Sized + 'a {
    fn extract_args(args: &'a [Value]) -> Result<Self, String>;
}

impl<'a, V: DowncastArg<'a>> ExtractArgs<'a> for V {
    fn extract_args(args: &'a [Value]) -> Result<Self, String> {
        match args {
            [value] => V::downcast_arg(value)
                .ok_or_else(|| format!("invalid argument {value:?}")),
            _ => Err("invalid argument count".to_owned()),
        }
    }
}

// Also impl for tuples:
// impl<'a, V1, V2, ...> ExtractArgs<'a> for (V1, V2, ...)
// where
//     V1: DowncastArg<'a>,
//     V2: DowncastArg<'a>,
//     ...
// { ... }

struct Function(Box<dyn Fn(&[Value]) -> Result<Value, String>>);

impl Function {
    fn call(&self, args: &[Value]) -> Result<Value, String> {
        self.0(args)
    }
}

fn create_function_raw<Func, Args>(f: Func) -> Box<dyn Fn(&[Value]) -> Result<Value, String>>
where
    Func: Fn(Args) -> Result<Value, String>,
    Func: 'static,
    for<'a> Args: 'a + ExtractArgs<'a>,
{
    let f = move |args: &[Value]| -> Result<Value, String> {
        let extracted_args = Args::extract_args(args)?;
        f(extracted_args)
    };
    Box::new(f)
}

fn create_function<Func, Args>(f: Func) -> Function
where
    Func: Fn(Args) -> Result<Value, String>,
    Func: 'static,
    for<'a> Args: 'a + ExtractArgs<'a>,
{
    Function(create_function_raw(f))
}

fn main() {
    let successor = create_function(|i: i64| Ok(Value::Int(i + 1)));
    assert_eq!(
        successor.call(&[Value::Int(1)]),
        Ok(Value::Int(2)),
    );

    let len = create_function(|s: &str| Ok(Value::Int(s.len() as i64)));
    assert_eq!(
        len.call(&[Value::String("abc".to_owned())]),
        Ok(Value::Int(3)),
    );
}

This is the compile error:

error: implementation of `ExtractArgs` is not general enough
  --> src/main.rs:90:15
   |
90 |     let len = create_function(|s: &str| Ok(Value::Int(s.len() as i64)));
   |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of `ExtractArgs` is not general enough
   |
   = note: `&str` must implement `ExtractArgs<'0>`, for any lifetime `'0`...
   = note: ...but it actually implements `ExtractArgs<'1>`, for some specific lifetime `'1`

For concrete types, it works if I replace these lines for<'a> Args: 'a + ExtractArgs<'a> with for<'a> &'a str: ExtractArgs<'a>, but I don't know of a way to express that for generic types.


Solution

  • Note that the Func: Fn(Args) -> Result<Value, String> type parameter to create_function is not bounded with a higher-ranked trait bound (HRTB). Furthermore, the type parameter Args is a single type (e.g. &'0 str), and not a higher-kinded type (for<'1> &'1 str). Because of this, the compiler does not realize that you need the closure to be generic over an arbitrary lifetime '1 of the &str arg, and instead assigns the &str arg a specific lifetime '0.

    If Rust had higher-kinded types, the solution would be to make Args generic over a lifetime, and bound Func with a HRTB: Func: for<'a> Fn(Args<'a>) -> Result<Value, String>. This would cause the compiler to infer the type of the closure argument correctly to be the higher-kinded type for<'1> &'1 str.

    In the absence of higher-kinded types (i.e. Rust today), there are two solutions you can consider:

    Creating your own Functor trait and types (playground)

    Create your own functor trait Funct and use it in create_function and create_function_raw:

    trait Funct {
        type Args<'a>: ExtractArgs<'a> + 'a;
    
        fn call<'a>(&self, args: Self::Args<'a>) -> Result<Value, String>;
    }
    
    fn create_function_raw<Func: Funct + 'static>(
        f: Func,
    ) -> Box<dyn Fn(&[Value]) -> Result<Value, String>> {
        let f = move |args: &[Value]| -> Result<Value, String> {
            let extracted_args = Func::Args::extract_args(args)?;
            f.call(extracted_args)
        };
        Box::new(f)
    }
    
    fn create_function<Func: Funct + 'static>(f: Func) -> Function {
        Function(create_function_raw(f))
    }
    

    And create and use your own functors (Successor and Len) instead of the closures:

    struct Successor;
    
    impl Funct for Successor {
        type Args<'a> = i64;
    
        fn call<'a>(&self, args: Self::Args<'a>) -> Result<Value, String> {
            Ok(Value::Int(args + 1))
        }
    }
    
    let successor = create_function(Successor);
    
    struct Len;
    
    impl Funct for Len {
        type Args<'a> = &'a str;
    
        fn call<'a>(&self, args: Self::Args<'a>) -> Result<Value, String> {
            Ok(Value::Int(args.len() as i64))
        }
    }
    
    let len = create_function(Len);
    

    Unfortunately implementing Funct for a generic closure Func: Fn(Args) -> Result<Value, String> would not work because of the same problem, and implementing it manually for different values of Args will not work without specialization. This approach becomes unwieldy, and though the boilerplate can be reduced with macros, it is not nearly as convenient as directly using closures.

    Rework the DowncastArg and ExtractArgs traits (playground)

    The second approach is to rework the DowncastArg and ExtractArgs traits so that ExtractArgs provides a generic associated type (GAT) ExtractArgs::Extracted<'a> that we can use as the closure argument type. While there are subtle differences to your approach (and certain niche cases/types where this one may not work), I would prefer this one if it works for all your use cases.

    trait DowncastArg {
        fn downcast_arg(value: &Value) -> Option<&Self>;
    }
    
    // impls
    
    trait ExtractArgs {
        type Extracted<'a>;
        
        fn extract_args<'a>(args: &'a [Value]) -> Result<Self::Extracted<'a>, String>;
    }
    
    impl<V: DowncastArg + ?Sized + 'static> ExtractArgs for V {
        type Extracted<'a> = &'a Self;
    
        fn extract_args<'a>(args: &'a [Value]) -> Result<Self::Extracted<'a>, String> {
            // Same impl
        }
    }
    

    This then allows us to use a HRTB for the closure argument to create_function:

    fn create_function<Func, Args>(f: Func) -> Function
    where
        Func: for<'a> Fn(Args::Extracted<'a>) -> Result<Value, String>,
        Func: 'static,
        Args: ExtractArgs + ?Sized,
    {
        Function(create_function_raw(f))
    }
    

    This does have the downside of the compiler being unable to infer the type of Args in a create_function invocation, and its value must be specified manually:

    let successor = create_function::<_, i64>(|i| Ok(Value::Int(*i + 1)));
    

    If you prefer, this can be worked around by moving create_function to the ExtractArgs trait so it can be called as an associated function of the ExtractArgs type (str::create_function(|s| Ok(Value::Int(s.len() as i64)))):

    trait ExtractArgs {
        // ...
        fn create_function(f: impl for<'a> Fn(Self::Extracted<'a>) -> Result<Value, String> + 'static) -> Function {
            Function(create_function_raw(f))
        }
    }
    

    Note: I made DowncastArg::downcast_arg return &Self for simplicity. If you want implementations to be able to return other types (like how your DowncastArg impl for i64 returns an i64), you could add a Downcasted<'a> GAT to DowncastArg and return it in downcast_arg instead, akin to the Extracted<'a> GAT in ExtractArgs.