Search code examples
pythoniopipegzippopen

How to feed standard output of a python program into Popen standard input?


At the moment, I have code to unzip a file and then zipping it back up in another file using Popen

with open('./file.gz', 'rb') as in_file:
    g_unzip_process = Popen(['gunzip', '-c'], stdin=in_file, stdout=PIPE)

with open('./outfile.gz', 'wb+') as out_file:
    Popen(['gzip'], stdin=g_unzip_process.stdout, stdout=out_file)

Now, I am trying to insert something in between.

It should look like this:

with open('./file.gz', 'rb') as in_file:
    g_unzip_process = Popen(['gunzip', '-c'], stdin=in_file, stdout=PIPE)

transform_process = Popen(['python', 'transform.py'], stdin=g_unzip_process.stdout, stdout=PIPE)

with open('./outfile.gz', 'wb+') as out_file:
    Popen(['gzip'], stdin=transform_process.stdout, stdout=out_file)

My transform.py code looks like this

for line in sys.stdin:
    sys.stdout.write(line)
sys.stdin.close()
sys.stdout.close()

After running it, why is it that my outfile.gz file is empty? I have also tried reading each line by running:

for line in transform_process.stdout:
    print(line)

How do I make it so that those lines will be written inside outfile.gz?


Solution

  • I was able to get this working by calling Popen.wait() for writing to outfile.gz