Search code examples
pythonpython-3.xsubprocessperfmon

How to put the live data in the queue from subprocess.Popen() stdout or communicate() function?


I'm new to subprocess module of python. I have below piece of code which executes the command on the windos machine and fetches the data from the windows perfmon counters. The command executes over the period of indefinite time. I want to add this data in queue. I tried the below ways but nothing seems to add the data stream in the queue. Not a single record

try1 - q.put(p.communicate())
try2 - q.put(p.communicate()[0])
try3 - q.put(p.communicate()[0].stdout.readline())
try4 - for line in p.stdout: q.put(line)

This trials does not return any errors but just dont add anything in the queue

q = Queue(maxsize=3)
t = threading.Thread(target=read_queue, args=(q,))
fields = []
cmd = ['TYPEPERF', '\Processor(*)\*', '-si', '10', '-sc', '5']
with Popen(cmd, bufsize=1, universal_newlines=True, shell=False) as p:
    count = 1
    t.start()
    output, err = p.communicate()
    print(output)
    if p.communicate()[0] is not None:
        print("adding data to the q ")
        q.put(p.communicate()[0].stdout.readline())
    else:
        p.terminate()

How to add the live data record by record in queue, command returns the comma separated data stream after every 5 sec. Please point if anything I am missing or where I'm going off track.

Note: I have looked at other threads on SO but couldn't find one to fix my problem.


Solution

  • After trying several attempts finally able to manage this, I was finding the q empty, because the first element in the queue is empty. Also the live stream will be stored based on stdout.PIPE That I used to iterate over and then pushed it to queue. before pushing the data to the queue check whether data is None and empty. Sample code -

    cmd = ['TYPEPERF', '\Processor(*)\*', '-si', '10', '-sc', '5']
    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True, shell=False)
    
    while True:
        line = proc.stdout.readline()
        if not line:
            break
        #print("test:", line.rstrip())
        if line.rstrip() is not None and line.rstrip() != '':
            print("test:", line.rstrip())
            q.put(line.rstrip())