Search code examples
rust

borrowed data escapes outside of closure, reassigning buffer to slice of itself in closure


Usually I understand why a borrowed data cannot escape the closure, but in this case I'm reassign output to a part of itself, so I'm not trying to assign it to something inside the closure, which would cause the escape.

What is happening?

struct A {
    a: u8
}

impl A {
    pub fn fill_and_write(
        &self,
        values: &[u8],
        mut output: &mut [u8],
    ) -> Result<usize, ()>{
        let len = output.len();
        values.iter().try_for_each(|v| {
            let written = 3; // write something and return how many was written
            output = &mut output[written..];
            Ok::<(),()>(())
        });
        Ok(len - output.len())
    }
}

playground

Error:

  --> src/lib.rs:14:13
   |
9  |         mut output: &mut [u8],
   |         ---------- `output` declared here, outside of the closure body
...
14 |             output = &mut output[written..];
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0597]: `output` does not live long enough

Solution

  • The reason is subtle.

    From the compiler point of view, &mut output and output = are two distinct operations. The first moves out of output, the second inserts a value back. And arbitrary things may happen between these: for example, what if written is out of bounds? The callback will panic, and output will remain uninitialized. Maybe someone will catch the panic and use the uninitialized output. This is bad.

    try_for_each() takes a FnMut closure, which means it cannot leave variables uninitialized (even temporarily, because of panics as we saw), i.e. it cannot consume them, or they cannot escape it. It is the same as if the closure holds just a mutable reference to output, and not an owned value (not coincidentally: FnOnce takes self by value, and so all captured variables are treated like they are owned. FnMut takes &mut self, and so all captured variables are treated like they were behind a mutable reference. And Fn takes &self, so all captured variables are treated like they were behind a shared reference).

    But there is another path to success: a reborrow. We have output: &mut &mut [u8], and we reborrow it to get &mut [u8].

    The problem of reborrow is that a reborrow of &'a mut &'b mut T can only give you a short-lived reference &'a mut T. So if we have output: &'b mut [u8], and we have a reference to it &'a mut &'b mut [u8], we can only get &'a mut [u8]. But we try to assign it to output, which is &'b mut [u8]; the lifetime does not live long enough. This is a similar problem to what one would face when trying to implement mutable iterators.

    And the fix is the same, too (without unsafe): temporarily take the value out of the reference, so you own it and have the full lifetime. That is:

    pub fn fill_and_write(&self, values: &[u8], mut output: &mut [u8]) -> Result<usize, ()> {
        let len = output.len();
        values.iter().try_for_each(|v| {
            let written = 3; // write something and return how many was written
            let my_output = std::mem::take(&mut output);
            output = &mut my_output[written..];
            Ok::<(), ()>(())
        });
        Ok(len - output.len())
    }