Search code examples
pythonpython-3.xargparsepython-exec

How to hide args to argparse when running script with exec?


I have 2 files, runner.py that runs target.py with subprocess or exec.
They both have command line options.

If runner runs target with subprocess it's ok:

$ python runner.py
run target.py with subprocess...
target.py: running with dummy = False

If runner runs target code with exec (with the -e option):

$ python runner.py -e
run target.py with exec...
usage: runner.py [-h] [-d]
runner.py: error: unrecognized arguments: -e

the command line argument -e is "seen" by target.py code (which accepts only one --dummy option) and raises an error.

How can I hide args to argparse when running script with exec?

Here's the code:

runner.py

import subprocess
import argparse


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("-e", "--exec", help="run with exec", action="store_true")
    args = parser.parse_args()

    target_filename = "target.py"

    if args.exec:
        print("run target.py with exec...")
        source_code = open(target_filename).read()
        compiled = compile(source_code, filename=target_filename, mode="exec")
        exec(compiled) # OPTION 1 - error on argparse
        # exec(compiled, {}) # OPTION 2 - target does not go inside "if main"
        # exec(compiled, dict(__name__="__main__")) # OPTION 3 - same error as OPTION 1 
    else:
        print("run target.py with subprocess...")
        subprocess.run(["python3", target_filename])

I tried to hide the globals with the commented options above, but without luck.
Seems related to how argparse works.

target.py

import argparse

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("-d", "--dummy", help="a dummy option", action="store_true")
    args = parser.parse_args()

    print(f"target.py: running with dummy = {args.dummy}")

Solution

  • There is the argparse conflict_handler option so one can write
    argparse.ArgumentParser(conflict_handler='resolve') in target script.
    resolve removes the conflicting options, but that doesn't handle well similar cases where the options have the same name both in runner and target or the case where you can't or don't want to change the target file.


    Here's the solution I have found.
    Internally argparse uses sys.argv to retrieve options set with command line.

    You can directly set sys.argv = [target_filename] which removes the options, but changing sys can give a lot of other problems.

    Using unittest.mock.patch (python3.4+) the sys.argv can be securely altered like this:

    from unittest.mock import patch
    
    # [...]    
    
    source_code = open(target_filename).read()
    compiled = compile(source_code, filename=target_filename, mode="exec")
    
    # remove command args
    with patch('sys.argv', [target_filename]):
        exec(compiled)
    

    So one can also run target script code with options:

    # run target
    with patch('sys.argv', [target_filename]):  
        exec(compiled)
    
    # run target with -d
    with patch('sys.argv', [target_filename, "-d"]):
        exec(compiled)