nvm works in bash, but not when executed in Python script

After installing nvm, it has been working normally until I need to use nvm to remotely connect to the server with python. There is no error or success in switching the node version, so I wrote a script on the server for testing:

test.py:

import os
res = os.popen("nvm use v14.2.0")
# res = os.popen("~/nvm/nvm.sh use v14.2.0")
print(res)
print(res.read())

output:

$ python3 test.py
<os._wrap_close object at 0x7f00acdc4ac8>
/bin/sh: nvm: command not found

The nvm installation path is ~/nvm and ~/nvm/nvm.sh

If I execute nvm -v directly on the server, the output is 0.39.1. When I execute nvm directly it works fine. Why does it not work when executed from the Python script?

Executing sh ./nvm.sh in the nvm installation directory does not give any output.

-rwxrwxr-x 1 asd asd 139220 Mar 11 15:57 nvm.sh

$ cd ~/nvm
$ sh ./nvm.sh
$

my .bashrc file:

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi

# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=

# User specific aliases and functions
source ~/nvm/nvm.sh

The core problem is that nvm can be executed on the command line, but the path where the nvm executable file is located cannot be found. Or nvm.sh is the executable but nothing happens when I execute this file.

NB: When I type which nvm it outputs:

/usr/bin/which: no nvm in (/home/pyer/nvm/versions/node/v14.2.0/bin:/usr/local/nodejs/bin:/usr/local/bin :/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lib/golang/bin:/usr/local/apache-maven-3.8.4/bin)

Update

After the last test, the code execution will no longer report errors or block, but new problems have been found. The code doesn't seem to execute successfully.I rewrote a test script

test.py:

import subprocess

def test():
    # p = subprocess.Popen(['bash', '-c', '-i', 'nvm use v14.2.0'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, executable='/usr/bin/bash')
    p = subprocess.Popen(['bash', '-c', '. ~/nvm/nvm.sh; nvm use v14.2.0'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, executable='/usr/bin/bash')
    out, err = p.communicate()
    print(out.decode('utf-8'))

    # p = subprocess.Popen(['bash', '-c', '-i', 'nvm list'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, executable='/usr/bin/bash')
    p = subprocess.Popen(['bash', '-c', '. ~/nvm/nvm.sh; nvm list'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, executable='/usr/bin/bash')
    out, err = p.communicate()
    print(out.decode('utf-8'))

test()

output:

$ python3 test.py 
Now using node v14.2.0 (npm v6.14.4)

         v8.9.0
->     v10.24.1
       v12.10.0
        v14.2.0
        v17.5.0
         system
default -> v14.2.0
iojs -> N/A (default)
unstable -> N/A (default)
node -> stable (-> v17.5.0) (default)
stable -> 17.5 (-> v17.5.0) (default)
lts/* -> lts/gallium (-> N/A)
lts/argon -> v4.9.1 (-> N/A)
lts/boron -> v6.17.1 (-> N/A)
lts/carbon -> v8.17.0 (-> N/A)
lts/dubnium -> v10.24.1
lts/erbium -> v12.22.10 (-> N/A)
lts/fermium -> v14.19.0 (-> N/A)
lts/gallium -> v16.14.1 (-> N/A)

It shows that v14.2.0 is being used, but it is actually the default v10.24.1. I found that nvm is a little different from other commands. On my machine, other commands have executable files. But nvm is loaded with .bashrc/.zshrc, so I have two guesses. 1 is because the bash called by python does not load .bashrc, so the nvm when python is executed is different from the nvm executed on the command line, maybe the scope is different? 2 is because python's subprocess calls some scripts, which will be executed in a sandbox for 'safety', such as nvm?

Solution

The error is pretty clear: the system cannot find the nvm command. This is probably because the search path in the subprocess is different the the one in your shell.

The documentation gives the following recommendation about this:

Warning: For maximum reliability, use a fully-qualified path for the executable. To search for an unqualified name on PATH, use shutil.which(). On all platforms, passing sys.executable is the recommended way to launch the current Python interpreter again, and use the -m command-line format to launch an installed module.

Resolving the path of executable (or the first item of args) is platform dependent. (...)

You could also just change the command to include the full path. So something like:

os.popen("/usr/bin/nvm use v14.2.0")

To find out the correct path, type which nvm in your shell. This should print the full path of your nvm executable.

Update

While the above is true for all applications in general, the question is why the nvm executable cannot be found in your path. The documentation explains that the ~/nvm/nvm.sh script should be called first to "load nvm". I originally suspected that this would just set the PATH variable so the nvm executable can be found, but looking at nvm.sh, it seems that the nvm command is actually not an executable but a shell function. That's why it cannot be found when trying to execute it from the Python script.

According to this answer, you should be able to run the command as follows:

subprocess.Popen(['bash', '-c', '. ~/nvm/nvm.sh; nvm'])

Update 2

Note that the nvm command seems to be intended to be used on the command-line. One of the things that the script does, is chaning environment variables such as your PATH. See here for example. That is how it would select a different version if you type nvm use v14.2.0.

If you're calling this from Python in the way you do, it does not have any effect. It would change the PATH variable in its current environment, which is then closed. Your second call to subprocess.Popen creates a new environment in which the default version will be used again. The following command should work as expected though, since both nvm commands will now be executed in the same shell process:

subprocess.Popen(['bash', '-c', '. ~/nvm/nvm.sh; nvm use v14.2.0; nvm list'],
                 stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                 executable='/usr/bin/bash')

However, you should really think about what you're trying to achieve here. Why are you calling nvm from a Python script? If you're just executing it to set the correct version, you might as well include that logic in your Python script. And if you're just calling a few nvm commands, you could easily do that in a bash script. However, if you really want to, you could do it this way, but you would have to remember the each call to subprocess.Popen() results in a new environment, similar to starting a new remote session to that server.