Search code examples
pythonbashsubprocesspopen

Using Popen in Python in ssh to Run "ls -lh INPUTS" Bash Commands

INPUTS is the variable I gave for the absolute path of a directory of possible input files. I want to check their status before going through my pipeline. So I tried:

import subprocess
import argparse

INPUTS = '/home/username/WinterResearch/Inputs'
status = subprocess.Popen(['ls', '-lh', INPUTS], shell=True, stdout=subprocess.PIPE)
stdout = status.communicate()
status.stdout.close()

I have also tried the often used

from shlx import split
import subprocess
import argparse

cmd = 'ls -lh INPUTS'
status = subprocess.Popen(cmd.split(), shell=True, stdout=subprocess.PIPE)

and 

cmd = "ls -lh 'INPUTS'"

I do not receive an error code. The process simply does not output anything to the terminal window. I am not sure why the python script simply skips over this instead of stating there is an error. I do receive an error when I include close_fds=True that states int cannot use communicate(). So how can I receive an output from some ls -lh INPUTS equivalent using subprocess.Popen()?

Solution

  • You don't see any output because you're not printing to console stdout — it's saved into a variable (named "stdout"). Popen is overkill for this task anyway since you aren't piping the command to another. check_output should work fine with subprocess for this purpose:

    import subprocess
subprocess.check_output("ls -lh {0}".format(INPUTS), shell=True)

    subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)
    Run command with arguments and return its output as a byte string.

    METHOD WITH LESSER SECURITY RISK: (see warnings plastered throughout this page)

    EDIT: Using communicate() can avoid the potential shell=True security risk:

    output = subprocess.Popen(["ls", "-lh", INPUTS]).communicate()[0]
print(output)