Calling a generic async function with a (mutably) borrowed argument

Question

Minimal example of my issue.

use std::future::Future;

async fn call_changer<'a, F, Fut>(changer: F)
where
    F: FnOnce(&'a mut i32) -> Fut,
    Fut: Future<Output = ()> + 'a,
{
    let mut i = 0;
    changer(&mut i).await; // error 1
    dbg!(i); // error 2
}

#[tokio::main]
async fn main() {
    call_changer(|i| async move {
        *i = 100;
    })
    .await;
}

This leads to two related errors, see rust playground for detailed output:

1. The borrow not living long enough because i gets dropped at the end of call_changer's body.
2. i cannot be used after the await because it is still being mutably borrowed.

I am a bit surprised by both, I understand why the Future return of F needs to have the same lifetime ('a) as its borrow (relevant async book section). However, according to that same reference, the borrow should be over as soon as I call the await on changer's result, which clearly does not happen or I would not have those errors. Reworking this example to something like the book where the changer function is not passed in as a parameter but just called directly works as expected.

What is going on here, and can I do anything about it? Replacing &mut with an Rc<RefCell<_>> construct works as expected, but if possible I'd like to avoid that.

Answer 1

TL;DR: Pin<Box<dyn Future>> is the only way to make it work currently:

use std::future::Future;
use std::pin::Pin;

async fn call_changer<F>(changer: F)
where
    F: FnOnce(&mut i32) -> Pin<Box<dyn Future<Output = ()> + '_>>,
{
    let mut i = 0;
    changer(&mut i).await; // error 1
    dbg!(i); // error 2
}

#[tokio::main]
async fn main() {
    call_changer(|i| {
        Box::pin(async move {
            *i = 100;
        })
    })
    .await;
}

---

When you specify 'a as a generic parameter, you mean "I permit the caller to choose any lifetime it wants". The caller may as well choose 'static, for example. Then you promise to pass &'a mut i32, that is, &'static mut i32. But i does not live for 'static! That's the reason for the first error.

The second error is because you're promising you're borrowing i mutably for 'a. But again, 'a may as well cover the entire function, even after you discarded the result! The caller may choose 'static, for example, then store the reference in a global variable. If you use i after, you use it while it is mutably borrowed. BOOM!

What you want is not to let the caller choose the lifetime, but instead to say "I'm passing you a reference with some lifetime 'a, and I want you to give me back a future with the same lifetime". What we use to achieve the effect of "I'm giving you some lifetime, but let me choose which" is called HRTB (Higher-Kinded Trait Bounds).

If you only wanted to return a specific type, not a generic type, it would look like:

async fn call_changer<'a, F, Fut>(changer: F)
where
    F: for<'a> FnOnce(&'a mut i32) -> &'a mut i32,
{ ... }

You can use Box<dyn Future> with this syntax as well:

use std::future::Future;
use std::pin::Pin;

async fn call_changer<F>(changer: F)
where
    F: for<'a> FnOnce(&'a mut i32) -> Pin<Box<dyn Future<Output = ()> + 'a>>,
{
    let mut i = 0;
    changer(&mut i).await;
    dbg!(i);
}

#[tokio::main]
async fn main() {
    call_changer(|i| {
        Box::pin(async move {
            *i = 100;
        })
    })
    .await;
}

Playground.

In fact, you can even get rid of the explicit for clause, since HRTB is the default desugaring for lifetimes in closures:

where
    F: FnOnce(&mut i32) -> &mut i32,

where
    F: FnOnce(&mut i32) -> Pin<Box<dyn Future<Output = ()> + '_>>,

The only question left is: How do we express this with generic Fut?

It's tempting to try to apply the for<'a> to multiple conditions:

where
    for<'a>
        F: FnOnce(&'a mut i32) -> Fut,
        Fut: Future<Output = ()> + 'a,

Or:

where
    for<'a> FnOnce(&'a mut i32) -> (Fut + 'a),
    Fut: Future<Output = ()>,

But both doesn't work, unfortunately.

What can we do?

One option is to stay with Pin<Box<dyn Future>>.

Another is to use a custom trait:

trait AsyncSingleArgFnOnce<Arg>: FnOnce(Arg) -> <Self as AsyncSingleArgFnOnce<Arg>>::Fut {
    type Fut: Future<Output = <Self as AsyncSingleArgFnOnce<Arg>>::Output>;
    type Output;
}

impl<Arg, F, Fut> AsyncSingleArgFnOnce<Arg> for F
where
    F: FnOnce(Arg) -> Fut,
    Fut: Future,
{
    type Fut = Fut;
    type Output = Fut::Output;
}

async fn call_changer<F>(changer: F)
where
    F: for<'a> AsyncSingleArgFnOnce<&'a mut i32, Output = ()>,
{
    let mut i = 0;
    changer(&mut i).await;
    dbg!(i);
}

Unfortunately, this does not work with closures. I don't know why. You have to put a fn:

#[tokio::main]
async fn main() {
    async fn callback(i: &mut i32) {
        *i += 100;
    }
    call_changer(callback).await;
}

Playground.

For more information:

• Lifetimes with async function parameters - users.rust-lang.org (the trait approach is from there).
• Generic for FnOnce that returns a future with a lifetime.
• How to talk about lifetime of opaque FnOnce result? - internals.rust-lang.org.

On nightly, this can be replaced with async closures, which this is very much the reason for their existence:

#![feature(async_closure)]

async fn call_changer<F>(changer: F)
where
    F: async FnOnce(&mut i32),
{
    let mut i = 0;
    changer(&mut i).await;
    dbg!(i);
}

#[tokio::main]
async fn main() {
    call_changer(async move |i| {
        *i = 100;
    })
    .await;
}