subprocess.CalledProcessError: returned non-zero exit status 1, while os.system does not raise any error

Question

Given the following command:

newman run tests.postman_collection.json -e environment.json  --reporters testrail,json,html

Raises:

RuntimeError: command 'newman run tests.postman_collection.json -e environment.json  --reporters testrail,json,html
' return with error (code 1): b'\nhttps://host.testrail.io/index.php?/runs/view/1234\n'

Py code that executes the command:

        try:
            newmanCLI_output = subprocess.check_output(npmCLi, shell=True).decode().strip()
        except subprocess.CalledProcessError as e:
            raise RuntimeError("command '{}' return with error (code {}): {}".format(e.cmd, e.returncode, e.output))

And yes I do use the check_output return.

The output is a url to test rail reports

Answer 1

That's a misfeature of os.system; it returns the exit code so you can examine it, but doesn't raise an error if something fails.

The check in subprocess.check_output means check that the command succeeded, or raise an exception otherwise. This is generally a good thing, as you don't want processes to die underneath you without a warning.

But you can work around it with subprocess.run if you want to disable it;

import shlex
result = subprocess.run(shlex.split(npmCLi), text=True, capture_output=True)
newmanCLI_output = result.stdout

The switch to avoid shell=True and use shlex.split to parse the string instead is not crucial, but hopefully demonstrates how to do these things properly.

You should still understand why exactly your command fails, and whether it is safe to ignore the failure.