Cannot exec() Python script from within entry_points script installed via pip and setuptools

Question

I followed these instructions for creating a setup.py file that would install a Python "executable" script. This is my project's structure:

pkgexec/
  setup.py
  pkgexec/
    __init__.py
    __main__.py
    core.py

As per the instructions, the __main__.py's main() method is an entry point in setup.py:

from setuptools import setup, find_packages

setup(
    name="pkgexec",
    version="0.2.0",
    packages=find_packages(),
    entry_points={ "console_scripts": ["pkgexec = pkgexec.__main__:main"]},
)

I installed the package from within the pkgexec/ directory, by running pip install -e ..

Until this point, everything works as expected.

What doesn't work is executing Python scripts via this "executable" entry point. You see, the whole aim of this package is to run Python scripts that import a bunch of stuff from the package, e.g. script.py uses functionality from the pkgexec package and is "ran" through the pkgexec "executable":

pkgexec script.py -v arg1 arg2

Here is the simplified version of __main__.py:

import argparse
import sys

from pkgexec import some_stuff

def main(args=None):
    if args is None:
        args = sys.argv[1:]

    parser = argparse.ArgumentParser()
    parser.add_argument('script', help='script to run via pkgexec')
    parser.add_argument(...)
    cli_args = parser.parse_args()

    print(f'{__name__}: Running script {cli_args.script}')
    exec(open(cli_args.script).read(), globals(), globals())  # <-- ???
    print(f'{__name__}: Done')


if __name__ == '__main__':
    sys.exit(main())

The problem: Nothing happens for exec(open(cli_args.script).read()) (tried it with and without , globals(), globals()). The script is not executed. What am I doing wrong here?

Workarounds that I'm not keen on:

• I can run the script via exec() if I don't "install" the entry point. Not an option.
• I can run the script if I import it via importlib. But this is too restrictive for users of the package that are supposed to write a main() method.

Answer 1

There's something happening with the script's globals when calling exec(). I can't figure out what runpy from the standard library is doing differently, but it's certainly working. The solution in my case is to replace the exec() call with a call to runpy.run_path().

Here is the modified __main__.py script for comparison:

import argparse
import os
import runpy
import sys

from pkgexec import some_stuff

def main(args=None):
    if args is None:
        args = sys.argv[1:]

    parser = argparse.ArgumentParser()
    parser.add_argument('script', help='script to run via pkgexec')
    parser.add_argument(...)
    cli_args = parser.parse_args()

    print(f'{__name__}: Running script {cli_args.script}')
    mod = argparse.Namespace(
        **runpy.run_path(cli_args.script,
                         run_name=os.path.basename(cli_args.script)))
    print(f'{__name__}: Done')


if __name__ == '__main__':
    sys.exit(main())

I'm sending the file name as run_name argument to run_path (this way the script "knows" its actual __name__ instead of the default <run_path> set by runpy).

Note that this solution is only applicable if the script to run does not contain a if __name__ == '__main__' section (which is precisely what I wanted).

UPDATE: The same effect can be achieved by doing what runpy does under the hood: compile() the code first, then exec() it afterwards:

# mod = argparse.Namespace(
#     **runpy.run_path(cli_args.script,
#                      run_name=os.path.basename(cli_args.script)))
code = compile(open(cli_args.script).read(),
               os.path.basename(cli_args.script), 
               'exec')
exec(code)