How do I run a shell script that requires the host Python environment from a script running in a virtual Python environment?

Question

I have a Python 3.9 script running in a virtual environment. The host environment is set up with Python 3.6 and many installed packages. I need to be able to run a shell script that only functions properly if it's run in the host environment.

It's not possible for me to make the 3.9 virtual environment work for this shell script, nor is it possible to update the shell script to work in Python 3.9.

When I run the script at the command line, I need to call deactivate, run the shell script, and then reactivate the virtual environment.

I'd like to be able to do the equivalent of that from within a Python program running in my 3.9 venv. But since I'm trying to run a shell script and not a Python program, I can't simply call the Python 3.6 interpreter directly to solve the problem.

I can think of two solutions:

1. Create a temporary shell script that deactivates the venv and calls the target shell script. Then run that temporary shell script from my Py3.9 program. I assume the deactivation will only apply to the scope I create when I run the shell script.
2. Create an alias for every program I want to run that first runs deactivate, e.g. alias run_myprog = "deactivate; myprog", although it would be kind of tedious to have to create one of these for every program I need to run.

Are there better solutions than the above two?

(This is related to this question. The difference is that question is asking about running a Python 3.6 program instead of a shell script that depends on Python 3.6.)

Answer 1

Activation of venv in a nutshell is just adding a few environment variables.

• VIRTUAL_ENV
• VIRTUAL_ENV_PROMPT

and modification of few other environment variables:

• PS1
• PATH

VIRTUAL_ENV variable points to the root of the venv. Depending on the OS you are running, what activation does is that it adds the value of ${VIRTUAL_ENV}/Scripts on windows and ${VIRTUAL_ENV}/bin on others like linux and mac to be the first element of your active PATH env.

Removing that will essentially be same as running without activation of venv - even thought its still activated for the parent process.

How to do that, depends on how you are calling your shell script. But for example, if you are using subprocess. Most if not all of its calling methods allow you to pass a new environment for the process and you can construct that with following snippet:

import os

deactivated_env = os.environ.copy()
venv_root = deactivated_env.pop("VIRTUAL_ENV")
paths = deactivated_env["PATH"].split(os.pathsep)

deactivated_path = os.pathsep.join([x for x in paths if not x.startswith(venv_root)])
deactivated_env["PATH"] = deactivated_path
deactivated_env.pop("VIRTUAL_ENV_PROMPT")

# subprocess.run( ...., env=deactivated_env)

with that, your shellscript should fall back into using the system wide python and its settings..