How do I embed my shell scanning-script into a Python script?

Iv'e been using the following shell command to read the image off a scanner named scanner_name and save it in a file named file_name

scanimage -d <scanner_name> --resolution=300 --format=tiff --mode=Color 2>&1 > <file_name>

This has worked fine for my purposes. I'm now trying to embed this in a python script. What I need is to save the scanned image, as before, into a file and also capture any std output (say error messages) to a string

I've tried

    scan_result = os.system('scanimage -d {} --resolution=300 --format=tiff --mode=Color 2>&1 > {} '.format(scanner, file_name))

But when I run this in a loop (with different scanners), there is an unreasonably long lag between scans and the images aren't saved until the next scan starts (the file is created as an empty file and is not filled until the next scanning command). All this with scan_result=0, i.e. indicating no error

The subprocess method run() has been suggested to me, and I have tried

with open(file_name, 'w') as scanfile:

    input_params = '-d {} --resolution=300 --format=tiff --mode=Color 2>&1 > {} '.format(scanner, file_name)
    scan_result = subprocess.run(["scanimage", input_params], stdout=scanfile, shell=True)

but this saved the image in some kind of an unreadable file format

Any ideas as to what may be going wrong? Or what else I can try that will allow me to both save the file and check the success status?

Solution

subprocess.run() is definitely preferred over os.system() but neither of them as such provides support for running multiple jobs in parallel. You will need to use something like Python's multiprocessing library to run several tasks in parallel (or painfully reimplement it yourself on top of the basic subprocess.Popen() API).

You also have a basic misunderstanding about how to run subprocess.run(). You can pass in either a string and shell=True or a list of tokens and shell=False (or no shell keyword at all; False is the default).

with_shell = subprocess.run(
    "scanimage -d {} --resolution=300 --format=tiff --mode=Color 2>&1 > {} ".format(
        scanner, file_name), shell=True)

with open(file_name) as write_handle:
    no_shell = subprocess.run([
        "scanimage", "-d", scanner, "--resolution=300", "--format=tiff",
            "--mode=Color"],  stdout=write_handle)

You'll notice that the latter does not support redirection (because that's a shell feature) but this is reasonably easy to implement in Python. (I took out the redirection of standard error -- you really want error messages to remain on stderr!)

If you have a larger working Python program this should not be awfully hard to integrate with a multiprocessing.Pool(). If this is a small isolated program, I would suggest you peel off the Python layer entirely and go with something like xargs or GNU parallel to run a capped number of parallel subprocesses.