Turn generic method into trait object safe method

Question

I would like to make an adapter that removes generic parameters (to produce a trait object) as in the example below.

use std::ops::Deref;

fn make_dyn_box<I, S>(iter_in: I)
where
    I: Iterator<Item = S>,
    S: Deref<Target = u8>,
{
    let mut iter_out = iter_in.map(
        |s| -> Box<dyn Deref<Target = u8>> {Box::new(s)}
    );
    take_dyn_box(&mut iter_out)
}

fn take_dyn_box<'a: 'b, 'b>(
    iter: &'a mut (dyn 'a + Iterator<Item = Box<dyn 'b + Deref<Target = u8>>>),
) { }

Is there a way to accomplish this without a heap allocation, using only safe code, and no external dependencies?

The below is an idea of what I want, but the borrow checker doesn't allow this.

use std::ops::Deref;

fn make_dyn<I, S>(iter_in: I)
where
    I: Iterator<Item = S>,
    S: Deref<Target = u8>,
{
    let mut item = None;
    let item = &mut item;
    let mut iter_out = iter_in.map(|s| -> &dyn Deref<Target = u8> {
        item.replace(s);
        Option::as_ref(item).unwrap()
    });
    take_dyn(&mut iter_out)
}

fn take_dyn<'a: 'b, 'b>(
    iter: &'a mut (dyn 'a + Iterator<Item = &'b (dyn 'b + Deref<Target = u8>)>),
) { }

Answer 1

One simple way to do it is to require that the input iterator return references. This compiles:

fn make_dyn<'b, I, S>(iter_in: I)
where
    I: Iterator<Item = &'b S>,
    S: Deref<Target = u8> + 'b,
{
    let mut iter_out = iter_in.map(|s| -> &dyn Deref<Target = u8> {
        s as _
    });
    
    take_dyn(&mut iter_out)
}

fn take_dyn<'a: 'b, 'b>(
    _iter: &'a mut (dyn 'a + Iterator<Item = &'b (dyn 'b + Deref<Target = u8>)>),
) { }

Note that it's good style to have iterator adapters return iterators that can be further manipulated:

fn make_dyn_<'b, I, S>(iter_in: I) -> impl Iterator<Item = &'b (dyn 'b + Deref<Target = u8>)>
where
    I: Iterator<Item = &'b S>,
    S: Deref<Target = u8> + 'b,
{
    iter_in.map(|s| -> &dyn Deref<Target = u8> {
        s as _
    })
}

(You could also define this as a generic struct that implements the Iterator trait.)

Now: if you don't want to require that the input iterator return references, then there's no way to return your new iterator.

What you're doing in the example code is creating a little buffer within your iterator, and returning references to it.

If that buffer is stored in your iterator struct, what you're trying to create is called a streaming iterator and can't be implemented currently. This extremely long blog post explains why; essentially, it would require large and hairy extensions to Rust's type system.

You can do something like this, if you rearrange your code to have users pass closures into your functions. Then you control when the closures get called, so you can store the returned values into a buffer and pass references to the buffer into the closure. But that's not as ergonomic as the usual Iterator interface.

This is sort of what you've done with your example code... I'm not exactly sure why it doesn't work. If you changed take_dyn to take a single &'b (dyn 'b + Deref<Target = u8>)> and called it repeatedly it should work though.