Search code examples
bashgreppipetail

Bash not waiting for full results after pipe but immediate action on direct output


When using grep -r "something" ., it will print to output as something gets found.

Whenever using a pipe afterwards to postprocess, it will wait though until having received all the output.

Is there some way / syntax for piping per line as it gets available?

Don't bother with improving the example, it really is just an example:

grep -r "something" . | grep -v "somethingelse"  # excluding somethingelse

The result will only be there after everything has been searched (because of the pipe |), but I'm looking for syntax which would respond on each line from grep -r "something" . as the "line becomes available".

It's possible to pipe to a program that waits for input and handles anything that reaches it, but using the pipe will only send data to it as it is done completely?

Does anyone have an idea?


Solution

  • With grep you can use --line-buffered option. If program does not have a similar option, you can use stdbuf program, e.g.:

    tail -f access.log | stdbuf -oL cut -d ' ' -f1 | uniq
    

    In this example stdbuf runs cut program with its output stream modified to be line buffered.