`subprocess.run` invokes an executable (not script) using the python interpreter when it's not necessary

Question

Related: After pakage a python to a exe with pyinstaller, why it can not run on a new computer?

I'm packaging a command line application with a thin GUI wrapper written in Python using pyinstaller. I received feedback from user that it's not running. Specifically, the error message says (partially redacted):

Fatal error in launcher: Unable to create process using '"C:\Users\DannyNiu\source\repos\AutoSub\whisper-root\Scripts\python.exe"  "C:\Users\DannyNiu\source\repos\AutoSub-Done\dist\main-batch\_internal\whisper-root\Scripts\whisper.exe" ...

I've got a venv in "whisper-root", and the error message seem to suggest that the executable file ("whisper.exe") is being run by the Python interpreter. The venv was in the folder "AutoSub", which I renamed to "AutoSub-Done" to induce the error.

Here's how I've been invoking the executable:

...
base = os.path.dirname(__file__)
...
whisper = base+"/whisper-root/Scripts/whisper.exe"
...
subprocess.run([whisper, ...arguments...], executable = whisper)
...

What am I supposed to do?

Answer 1

In a venv, only package managers such as pip would add programs into the "Scripts" directory (where the whisper thing resides). And when pip does that, interpreter stubs are added to them. On most POSIX implementations, this is done by inserting the shebang (#!/interpreter/path) as the 1st line into the text scripts; on Windows, these scripts are blessed into executable files by virtue of some compilation mechanism which embeds the absolute path of the Python interpreter program from the venv.

This has 2 implications:

1. all programs in the venv depends on the absolute path of the venv python interpreter.
2. being in venv, the python interpreter itself (on windows) is a blessed redirect stub to the interpreter program installed in the operating system.

This is why it breaks so terribly when moved to another computer.

What I did to work around it, is to

1. install the openai-whisper program in an venv,
2. install an instance of python in the user directory,
3. copy the python installation into the packaging directory,
4. copy the content of site-packages directory into Libs in the Python instance in the packaging directory.
5. Write and compile a minimal EXE program to invoke the main python script using the python interpreter in the packaging root - absolute paths are found relative to this minimal program. It turns out it's easy to do this in C# - a language with memory-safe facilities to process strings and arrays. The only thing is to remember to provide a C# desktop runtime redistributable package.