Search code examples
pythonlinuxpython-venv

Can I use symlink to a Python virtual environment in my shebang statement?

I have set up a python virtual environment on a new linux server. I have a number of existing scripts I'd like to avoid changing (for various reasons) - these all have a shebang which points to a python installation not on the new server.

My hope was that I could create the same directory structure on the new server as well as a symlink on the new server that points to the new venv. If I specify the venv python installation in the shebang, my script runs correctly, recognizing the packages I installed, however if I run it with the symlink in the shebang, the script does not use the venv and fails.

This script uses a symlink I created using ln -s: /opt/{path on old server}/python -> /opt/test/test_venv368/bin/python3

Failing Script:

#!/opt/{path on old server}/python
import sys
print(sys.executable)
import pyodbc
modulenames = set(sys.modules) & set(globals())
allmodules = [sys.modules[name] for name in modulenames]
print(allmodules)

Outputs:

/opt/{path on old server}/python
Traceback (most recent call last):
  File "./testvenv3.py", line 4, in <module>
    import pyodbc
ModuleNotFoundError: No module named 'pyodbc'

Successful when not using a symlink:

#!/opt/{venv path}/python
import sys
print(sys.executable)
import pyodbc
modulenames = set(sys.modules) & set(globals())
allmodules = [sys.modules[name] for name in modulenames]
print(allmodules)

Outputs:

/opt/test/test_venv368/bin/python
[<module 'sys' (built-in)>, <module 'pyodbc' from '/opt/test/test_venv368/lib64/python3.6/site-packages/pyodbc.cpython-36m-x86_64-linux-gnu.so'>]

I have checked the permissions on and recreated the symlink. The sticking point to me is that the script runs correctly with the venv as the shebang and not when the symlink is used.

I'm also open if there is a way to force use of a specific python installation via the user profile. (As long as the script can remain untouched.) Having the user profile start with an activated venv doesn't work when running the script as an executable (which it needs to be).

Solution

  • I figured out what was happening:

    Both scripts were using the same installation of Python (in the venv), however launching the first/failing executable script even within an activated venv did not correctly locate the modules in the venv's site-packages folder.

    To fix this, we set the PYTHONPATH environment variable by adding export PYTHONPATH=/opt/test/test_venv368/lib/python3.6/site-packages/ to the user profile.

    Printing sys.path inside each run of the script showed differences in where it was looking for modules.