I am trying to read a live sensor stream using subprocess in python. In particular, subprocess runs a command line program that outputs the data. This data is input into STDOUT and then read line by line. here is the code:
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, bufsize=0)
while True:
line_out = proc.stdout.readline()
sys.stdout.flush()
This code produces sensor data output, but incurs a time lag as it runs. So initially the sensor and subprocess are in sync but within an hour, the subprocess is as much as 45 minutes behind.
As you can see I tried "bufsize=0". I also tried running python with the -u option. Pexpect is not available to me since I am running Windows 8 and python 2.7.
How can I get streaming output from subprocess that does not incur a time delay?
Please refer to this SO query that should address your problem. Based on the way you are creating the subprocess; you are using un-buffered input. This has poor performance not only in Python but in another runtime as well.
Set the bufsize to -1 - this will make sure that the full buffering (system default mode) is used to read your input.
Please check the performance after making this change and let us know.