Search code examples
linuxbashawkpipetail

Awk, pipe and tail -f giving unexpected behavior

Here is my sample log file.http://pastebin.com/DwWeFhJk.

When I am doing

tail -f log | awk '{if (NF>3) {print $1}; }'

the result I am getting is correct 

64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10

But when I am doing:

tail -f log |
awk '{if (NF>3) {print $1}; }' |
awk '{print $1}'

I am not getting any output. Even no output in case of

tail -f log | awk '{if (NF>3) {print $1}; }' | grep "64"

I am not getting the reason why the output of the first awk is not getting passed as the input of the second awk/grep after the pipe.

Solution

  • When the output of the first awk is going to the terminal, the output is line-buffered, so each line is printed as it is produced. When the output is going to the second awk or the grep, it is fully buffered. The output won't be sent until the buffer is full. When enough extra records are appended to the log, then the second awk will a buffer full of data to process. Until then, nothing will happen.