Search code examples
rustclosureslifetimeownership

Why `FnMut` closure does not allow shared reference to captured variable to escape?


I'm trying to understand why a shared reference &s doesn't work here. As of my understanding, the closure passed to map owns s so it should be ok to return something that has shared reference to s. It does not try to move s multiple times or produce multiple mutable references here. So why the error? Sorry if the code doesn't seem to make sense, it's just used to serve as an example!

    fn take_ref_produce_impl_trait(s: &str) -> impl IntoIterator<Item=String> + '_ {
        (0..s.len()).map(move |it| s[it..it+1].to_string())
    }

    fn produce_stream() {
        let s = "bbb".to_string();
        (1..3)
            .map(move |_| {
                take_ref_produce_impl_trait(&s)
            });
    }
error: captured variable cannot escape `FnMut` closure body
  --> src\experiments\closure.rs:61:17
   |
58 |         let s = "bbb".to_string();
   |             - variable defined here
59 |         (1..3)
60 |             .map(move |_| {
   |                         - inferred to be a `FnMut` closure
61 |                 take_ref_produce_impl_trait(&s)
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^-^
   |                 |                            |
   |                 |                            variable captured here
   |                 returns a reference to a captured variable which escapes the closure body
   |
   = note: `FnMut` closures only have access to their captured variables while they are executing...
   = note: ...therefore, they cannot allow references to captured variables to escape

----- UPDATE -----

Sorry for this but here is another example which lead to similar error, in fact, the example above would report the same error if produce_stream try to return the thing it dropped.

So if the reason that lead to the previous error was because the closure can be dropped before the iterators, then would it be for the same reason in this case?

If Then is the stream and its field f is the closure we passed to then, then the closure is not dropped while the stream try to produce its items. However the item is a future, which is placed in the future field. Is it because the future can be evaluated after Then is dropped?

The Then struct

pin_project! {
    /// Stream for the [`then`](super::StreamExt::then) method.
    #[must_use = "streams do nothing unless polled"]
    pub struct Then<St, Fut, F> {
        #[pin]
        stream: St,
        #[pin]
        future: Option<Fut>,
        f: F,
    }
}

Example

    use futures::{StreamExt, Stream};
    use futures::{stream};

    async fn take_ref_produce_impl_trait_v2(s: &str) -> impl Stream<Item=String> + '_ {
        stream::iter(0..s.len()).map(move |it| s[it..it+1].to_string())
    }

    async fn produce_stream_v2() -> impl Stream<Item=String> {
        let s = "bbb".to_string();
        stream::iter(1..3)
            .then(move |_| {
                take_ref_produce_impl_trait_v2(&s)
            })
            .flatten()
    }
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
  --> src\experiments\closure.rs:73:48
   |
73 |                 take_ref_produce_impl_trait_v2(&s)
   |                                                ^^
   |
note: first, the lifetime cannot outlive the lifetime `'_` as defined on the body at 72:19...
  --> src\experiments\closure.rs:72:19
   |
72 |             .then(move |_| {
   |                   ^^^^^^^^
note: ...so that closure can access `s`
  --> src\experiments\closure.rs:73:48
   |
73 |                 take_ref_produce_impl_trait_v2(&s)
   |                                                ^^
   = note: but, the lifetime must be valid for the static lifetime...
note: ...so that return value is valid for the call
  --> src\experiments\closure.rs:69:37
   |
69 |     async fn produce_stream_v2() -> impl Stream<Item=String> {
   |                                     ^^^^^^^^^^^^^^^^^^^^^^^^

Solution

  • Suppose you .collect()ed the iterator in produce_stream instead of dropping it. Then you would have a collection of impl IntoIterator<Item=String> + '_.

    But those iterators, with their inferred '_ lifetime, borrow from s, which is dropped when produce_stream's map closure is. That is, the closure is returning references to a string it owns, but the references outlive the closure.

    If you change the closure to not be a move closure, then it should compile because the closure and the overall iterator now borrows from the local variable s.

    But you won't be able to return that iterator from produce_stream, if that's what you were hoping to do; in that case you'd need to arrange for take_ref_produce_impl_trait to return an iterator that owns the string data it's accessing (or perhaps an Rc of it), so that the iterator has no lifetime requirement.