Search code examples
pythonsubprocessexternal-processpiping

Piping data from python to an external command


I've read everything there is on subprocess.Popen but I think I'm missing something.

I need to be able to execute a unix program which reads a data stream from a list created in the python script and write the result of that program to a file. From the bash prompt I do this all the time with no problem but now I am trying to to this from within a python script which preprocesses some binary files and a lot of data before coming to this stage.

Lets look at a simple example not including all the preprocessing:

import sys
from pylab import *
from subprocess import *
from shlex import split

# some arbitrary x,y points
points = [(11,31),(13,33),(15,37),(16,35),(17,38),(18,39.55)]

commandline = 'my_unix_prog option1 option2 .... > outfile'
command = split(commandline)

process = Popen(command, stdin=PIPE, stdout=PIPE, stderr=PIPE)
print process.communicate(str(points))

The way this would be executed in bash is:

echo "11 31
      13 33
      15 37
      16 35
      17 38
      18 39.55" | my_unix_prog option1 option2 .... > outfile

The way the data is fed into the unix prog is important as well, I should be formatted in 2 columns separated by whitespace.

Any help is appreciated...


Solution

  • SOLVED!

    With the help of Dhara and xhainingx I was able to figure this out:

    import sys
    from pylab import *
    from subprocess import *
    from shlex import split
    
    # some arbitrary x,y points
    points = [(11,31),(13,33),(15,37),(16,35),(17,38),(18,39.55)]
    
    commandline = 'my_unix_prog option1 option2 ....'
    command = split(commandline)
    
    process = Popen(command, stdin=PIPE, stdout=open('outfile', 'w'), stderr=PIPE)
    for p in points:
        process.stdin.write(str(p[0]) + ' ' + str(p[1]) + '\n')
    
    print process.communicate()
    

    This works very well, thanks.