Search code examples
arraysrustiobufferedreader

Put a mutable buffer of u8 into BufReader.read()


Follows on from this partial answer to my question there.

I'm basically trying to understand how a particular manifestation of BufReader works, which is not std::io::BufReader.

The puzzling thing is that (std::io::) BufReader::new(pipe.clone()) produces an object which is said to be type std::io::buffered::bufreader::BufReader<ipipe::pipe_windows::Pipe>. Clearly that's not the same as std::io::BufReader, which presumably explains why it doesn't implement trait std::io::BufRead, and thus the method lines is not available.

I couldn't find any documentation on the std::io::buffered::bufreader module. But its BufReader appears to implement trait Read, and thus a method read, I think. I'm assuming that in this context this is a blocking method, i.e. it waits until some data comes down the pipe (see details for std::io::Reader::read).

But I can't work out how to submit an array of u8 which the compiler will accept for this method: these are some of my unsuccessful attempts:

    let br = BufReader::new(pipe.clone());
    let buf = br.buffer();
    println!("buf {:?}", buf);
    // let mut v:[u8] = Vec::new();
    // let mut receiving_buf: [u8] = [0 as u8; 1000];
    // let mut receiving_buf = vec![0; 24];
    let mut receiving_buf = [0; 24];
    buf.read(&mut receiving_buf);

... I don't care whether it's on the heap or the stack... I just want the thing to work.

NB message signature of std::io::Reader::read is fn read(&mut self, buf: &mut [u8]) -> Result<usize>.


Solution

  • The puzzling thing is that (std::io::) BufReader::new(pipe.clone()) produces an object which is said to be type std::io::buffered::bufreader::BufReaderipipe::pipe_windows::Pipe. Clearly that's not the same as std::io::BufReader

    It is. That's just the internal name, which is then re-exported as std::io::BufRead.

    Here's the definition

    Here's the where it's re-exported to std::io::buffered

    here's where that's re-exported to std::io

    Which presumably explains why it doesn't implement trait std::io::BufRead, and thus the method lines is not available.

    No. The compiler literally tells you why there's no lines method in the original code:

    help: the following trait is implemented but not in scope; perhaps add a `use` for it:
         |
    1    + use std::io::BufRead;
    

    The traits need to be in-scope for their implementations to be visible. And while there are traits which are always in-scope (those exported in the prelude) that's not the case of BufRead.

    But I can't work out how to submit an array of u8 which the compiler will accept for this method: these are some of my unsuccessful attempts:

    I have no idea what that means. BufReader::buffer is the internal data buffer, it's where the BufReader puts the data it read from the wrapped Read: the point of a buffered reader is to "coalesce" reads by reading more than specifically requested, filling an internal buffer, and yielding content from there afterwards until the buffer is empty.

    That is because if say you're reading a byte at a time (for a streaming parser for instance) performing a read() syscall for every single byte is ungodly expensive. The BufReader will instead perform a larger read than that, then yield a single byte, and future single-byte reads will receive data from the bufreader's internal buffer until it's exhausted which is very cheap (pretty much just a bounds check).