How to ensure data is received between commands

Question

I'm using Paramiko to issue a number of commands and collect results for further analysis. Every once in a while the results from the first command are note fully returned in time and end up in the output for the second command.

I'm attempting to use recv_ready to account for this, but it is not working, so I assume I am doing something wrong. Here's the relevant code:

pause = 1

def issue_command(chan, pause, cmd):
    # send commands and return results
    chan.send(cmd + '\n')
    while not chan.recv_ready():
        time.sleep(pause)
    data = chan.recv(99999)

ssh = paramiko.SSHClient()
ssh.load_system_host_keys()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
chan = ssh.connect(host, port=22, username=username, password=password, timeout=3,)

resp1 = issue_command(chan, pause, cmd1)
resp2 = issue_command(chan, pause, cmd2)

The output for these commands is relatively small (a few sentences). Increasing the pause would likely solve the problem but is not an ideal solution.

Answer 1

I would use transport directly and create a new channel for each command. Then you can use something like:

def issue_command(transport, pause, command):
    chan = transport.open_session()
    chan.exec_command(command)

    buff_size = 1024
    stdout = ""
    stderr = ""

    while not chan.exit_status_ready():
        time.sleep(pause)
        if chan.recv_ready():
            stdout += chan.recv(buff_size)

        if chan.recv_stderr_ready():
            stderr += chan.recv_stderr(buff_size)

    exit_status = chan.recv_exit_status()
    # Need to gobble up any remaining output after program terminates...
    while chan.recv_ready():
        stdout += chan.recv(buff_size)

    while chan.recv_stderr_ready():
        stderr += chan.recv_stderr(buff_size)

    return exit_status, stdout, stderr

ssh = paramiko.SSHClient()
ssh.load_system_host_keys()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(host, port=22, username=username, password=password, timeout=3,)
transport = ssh.get_transport()
pause = 1    

resp1 = issue_command(transport, pause, cmd1)
resp2 = issue_command(transport, pause, cmd2)

An even better way would be to take a list of commands and spawn a new channel for each, poll each chan's recv_ready, and suck up their stdout/stderr when output is available. :-)

Edit: There are potential issues with reading data after the command exits. Please see the comments!