Read portion of lines from child process in Rust (chunk of data)

When I try to spawn a child ffmpeg process I use additonal flag -progress, next I use pipe to pass this progress output to the stderr. So the whole command looks like:

ffmpeg -i ... -progress pipe:2 ...

Without -progress flag ffmepg outputs following line in stderr, probably once per second:

frame=46 46 fps=0.0 q=0.0 size=       0kB time=00:00:01.72 bitrate=   0.2kbits/s speed=2.69x

With -progress flag ffmepg outputs (multiple lines) in stderr, probably once per second:

frame=1   1 fps=0.0 q=0.0 size=       0kB time=00:00:00.19 bitrate=   2.0kbits/s speed=2.94x    
fps=0.00
stream_0_0_q=0.0
bitrate=   2.0kbits/s
total_size=48
out_time_us=192000
out_time_ms=192000
out_time=00:00:00.192000
dup_frames=0
drop_frames=0
speed=2.94x
progress=continue

The main puppose of using -progress flag is to calc percentage of completion by parsing out_time_ms line and comparing to the whole duration.

Reading this chunk (portion of lines) is pretty simple in NodeJS:

const { spawn } = require('child_process');
const child = spawn('ffmpeg', [..., '-progress', 'pipe:2', ...]);

child.stderr.on('data', (data) => {
  // data will contain multiple lines, exactly one chunk
});

Reading this chunk (portion of lines) is pretty simple in Deno also:

const child = Deno.spawnChild("ffmpeg", {
  args: [..., '-progress', 'pipe:2', ...],
});
const stderrReader = child.stderr.getReader();
let readResult = await stderrReader.read();
while (readResult.done !== true) {
  readResult = await stderrReader.read();
  // readResult will contain multiple lines, exactly one chunk
}

I can't achieve the same in rust:

let mut command = Command::new("ffmpeg");
command.args(["...", "-progress", "pipe:2", "..."]);
let mut child = command
  .stdout(Stdio::piped())
  .stderr(Stdio::piped())
  .spawn()
  .unwrap();

let child_stderr = child.stderr.as_mut().expect("Unable to pipe stderr");
let mut reader = BufReader::new(child_stderr);
let mut buff = String::new();
while reader.read_line(&mut buff).expect("Unable to read chunk") > 0 {
  // buff will contain only on line
  buff.clear();
}

I am new in Rust. I can not detect what character signals end of chunk.

Runnig read_line() - will read only one line.
Runnig read_to_end() - will read the whole output until the end of process (EOF).

How can I read in Rust portion of lines that ffmpeg outputs probably once per second? How Node/Deno detects this "end of chunk"? Does rust have such event/signal also?

Solution

You can use ChildStderr::read to achieve the same behaviour as in the other languages.

⚠ WARNING ⚠
This behaviour is not reliable, in any language.

Stderr is a stream of data. It is not packet-based. It is meant to be consumed character by character.

The reason why you do get chunks is for performance reasons. The operating system groups multiple characters together to a single read/write operation. Note that the decision which characters to group is arbitrarily chosen by the operating system.

In your case, it just looks as if this status info always comes as one packet, because it gets produced so fast that it's smart of the operating system to group it together. Note, however, that there is no guarantee that this is always the case and relying on it is dangerous.

BufReader abstracts all of that away and exposes common functionalities like reading a line of text.

Recommendation:
Use BufReader to process the status information line by line. There is no reliable way to detect the end of the status update, so process each line as a self-contained message. (unless ffmpeg prints some kind of end-of-message marker, like a double newline)