I want to read a stream of values for multiple files on disc. These might be CSV files, or tab-separated, or some proprietary binary format. Therefore I want my function that handles reading multiple files to take the Path -> Iterator<Data> function as an argument. If I understand correctly, in Rust I need to box the iterator, and the function itself, since they're unsized. Therefore my reading function should be (I'm just using i32 as a simple proxy for my data here):
fn foo(read_from_file: Box<dyn Fn(&Path) -> Box<dyn Iterator<Item=i32>>>) {
panic!("Not implemented");
}
For testing, I'd rather not be reading actual files from disc. I'd like my test data to be right there in the test module. Here's roughly what I want, but I've just put it into the main of a bin project for simplicity:
use std::path::Path;
fn foo(read_from_file: Box<dyn Fn(&Path) -> Box<dyn Iterator<Item=i32>>>) {
panic!("Not implemented");
}
fn main() {
let read_from_file = Box::new(|path: &Path| Box::new(match path.as_os_str().to_str().unwrap() {
"/my_files/data.csv" => vec![1, 2, 3],
"/my_files/data_2.csv" => vec![4, 5, 6],
_ => panic!("Invalid filename"),
}.into_iter()));
foo(read_from_file);
}
This gives me a compilation error:
Compiling iter v0.1.0 (/home/harry/coding/rust_sandbox/iter)
error[E0271]: type mismatch resolving `for<'r> <[closure@src/main.rs:9:35: 13:19] as FnOnce<(&'r Path,)>>::Output == Box<(dyn Iterator<Item = i32> + 'static)>`
--> src/main.rs:15:9
|
15 | foo(read_from_file);
| ^^^^^^^^^^^^^^ expected trait object `dyn Iterator`, found struct `std::vec::IntoIter`
|
= note: expected struct `Box<(dyn Iterator<Item = i32> + 'static)>`
found struct `Box<std::vec::IntoIter<{integer}>>`
= note: required for the cast to the object type `dyn for<'r> Fn(&'r Path) -> Box<(dyn Iterator<Item = i32> + 'static)>`
For more information about this error, try `rustc --explain E0271`.
error: could not compile `iter` due to previous error
I don't really understand this. Doesn't std::vec::IntoIter implement Iterator, in which case I don't see why this is a type error?
If I add an explicit type annotation Box<dyn Fn(&Path) -> Box<dyn Iterator<Item=i32>>>, this compiles:
use std::path::Path;
fn foo(read_from_file: Box<dyn Fn(&Path) -> Box<dyn Iterator<Item=i32>>>) {
panic!("Not implemented");
}
fn main() {
let read_from_file : Box<dyn Fn(&Path) -> Box<dyn Iterator<Item=i32>>>
= Box::new(|path: &Path| Box::new(match path.as_os_str().to_str().unwrap() {
"/my_files/data.csv" => vec![1, 2, 3],
"/my_files/data_2.csv" => vec![4, 5, 6],
_ => panic!("Invalid filename"),
}.into_iter()));
foo(read_from_file);
I'm very confused by why this works. My understanding of Rust is that, in a let definition, the explicit type is optional - unless the compiler cannot infer it, in which case the compiler should emit error[E0283]: type annotations required.
To me, this reads like a failure of type inference, since the closure is unable to infer that it needs to return a pointer to a v-table (from dyn Iterator).
However, I'd suggest that Box<dyn Foo> might not be necessary here. It's true that, since Iterator is a trait, you can't know the size of it at compile-time, in a sense, you can.
Rust "monomorphizes" generic code, which means it generates copies of generic functions/structs/etc for each concrete type it is used with. For example, if you have:
struct Foo<T> {
value: T
}
fn main() {
let _ = Foo { value: "hello" };
let _ = Foo { value: 123 };
}
It's going to generate a Foo_str_'static and a Foo_i32 (roughly speaking) and substitute those in as needed.
You can exploit this to use static dispatch with generics while using traits. Your function can be rewritten as:
fn foo<F, I>(read_from_file: F)
where
F: Fn(&Path) -> I,
I: Iterator<Item = i32>,
{
unimplemented!()
}
fn main() {
// note the lack of boxing
let read_from_file = |path: &Path| {
// ...
};
foo(read_from_file);
}
This code is (very probably but I haven't benchmarked) faster, and more idiomatic, as well as making the compiler error go away.