I have a program that when it runs, outputs hundreds of lines starting with "Info:" and a few lines that have useful output. To make things easier, I have created a simple python and bash combination script to emulate the issue I'm having:
wait_2sec.py:
import time
print("Hello!")
time.sleep(2)
print("Goodbye!")
I am attempting to trim my output by running:
python wait_2sec.py | sed '/Goodbye/d'
However, sed does not output Hello! until after the python script has finished. I don't know whether the pipe waits until after the program is finished to begin running the sed command, or if the sed command is the hangup. I am open to using another command to trim output if sed does not work for this use-case.
You can try to run your script using the following command:
unbuffer python wait_2sec.py | sed '/Goodbye/d'
Refrence: https://unix.stackexchange.com/a/200413