IndexError: list index out of range when trying to run cProflie

Question

I have a main.py script in my project that runs a model which is organized into a python package. The project structure looks like this:

project/
    model/
        data/
            ...
        __init__.py
        ...
    tests/
        ...
    main.py
    integrator.py
    utils.py

When I run main.py directly i.e. python main.py it works fine. However, I am trying to profile main.py using:

python -m cProfile -o main.py

and get the following error:

Traceback (most recent call last):
  File "C:\Users\pbreach\Anaconda3\lib\runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\pbreach\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\pbreach\Anaconda3\lib\cProfile.py", line 160, in <module>
    main()
  File "C:\Users\pbreach\Anaconda3\lib\cProfile.py", line 155, in main
    parser.print_usage()
  File "C:\Users\pbreach\Anaconda3\lib\optparse.py", line 1587, in print_usage
    print(self.get_usage(), file=file)
  File "C:\Users\pbreach\Anaconda3\lib\optparse.py", line 1573, in get_usage
    self.expand_prog_name(self.usage))
  File "C:\Users\pbreach\Anaconda3\lib\optparse.py", line 1550, in expand_prog_name
    return s.replace("%prog", self.get_prog_name())
  File "C:\Users\pbreach\Anaconda3\lib\optparse.py", line 1545, in get_prog_name
    return os.path.basename(sys.argv[0])
IndexError: list index out of range

I am aware of the difference between python module.py and python -m module to the extent of this answer (not very)... But I'm not sure what I'm doing wrong here. From the traceback it seems like not enough arguments were passed to the script, but I'm not that familiar with cProfile.

Any ideas as to why this error is coming about? How can I get the code to profile?

Answer 1

You're using the -o flag, that means it's expecting you to provide an output file after the flag. It also means that sys.argv won't include main.py Try this instead: python -m cProfile -o output.txt main.py

For clarity:

$ python -m cProfile -o output.txt main.py
# |-----------------|
# This part is the command


$ python -m cProfile -o output.txt main.py
#                    |------------|
# this is a named argument to the command.


$ python -m cProfile -o output.txt main.py
#                                 |-------|
# this is a positional argument to the command.