how to use pexpect to get spontaneous output of subprocess in python

Question

This is related to my another post multithreading issue with wx.TextCtrl (or underlying GTK+), which after correction with calling GUI interactions from primary thread, I find it again comes to the pipe block buffering problem. so How to get spontaneous output from the subprocess.stdout?

To be in short, currently I am using subprocess.popen to launch an external long-time running program.

    launchcmd=["EXTERNAL_PROGRAM_EXE"]
    p = subprocess.Popen(launchcmd, stdin=subprocess.PIPE, 
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    self.outputThread = BashProcessThread(p.stdout.readline)
    self.outputThread.start()
    # wx.TextCtrl is used to make input/output
    self.textctrl = wx.TextCtrl(self, style=wx.TE_PROCESS_ENTER|wx.TE_MULTILINE)

And I use a separate thread to read the stdout of the background program, with "wx.CallAfter" to call back.

class BashProcessThread(threading.Thread):
    def __init__(self, readlineFunc, textctrl):
        threading.Thread.__init__(self)
        self.readlineFunc = readlineFunc

    def run(self):
        while True:
           line = self.readlineFunc()
           wx.CallAfter(textctrl.AppendText(line))

The above code prints out the subprocess log messages block-hanging-block (instead of spontaneously line by line), and the worst is the remaining 5-6 lines of log messages could not be timely printed until the user send the next input.

From my old post, I get to know there is pty and pexpect, which could make the subprocess thought it is interacting with pseudo-tty. But how should pexpect be used, especially considering the background process is long-term, independent running task?

e.g., If I used

child=pexpect.spawn(launchcmd)

How can I get the output and input of the subprocess, so I could use wx.TextCtrl to print the output, and also use wx.TextCtrl to forward user input to subprocess?

Answer 1

Have you tried something like:

child = pexpect.spawn(launchcmd)
while True:
    try:
        child.expect('\n')
        print(child.before)
    except pexpect.EOF:
        break