I was trying to use a simple script that imported the library namespaces when using SLURM and sbatch however, I am not able to do it because it doesn't find the library (even though pip list shows its installed in my environment).
The script I am running is ridiculously simple:
#!/bin/python
#SBATCH --job-name=Python
#import namespaces as ns
import os
#location it should be installed
path = '/home/username/envs/tensorflow/lib/python2.7/site-packages'
ls = os.listdir(path)
print(ls)
print('namespaces' in ls) #does exist
#when using sbatch these lines fail
import namespaces as ns
print ns
it simply shows where the library is installed. When I run it on the head node of SLURM (the cluster) with python ns_test.py it prints correctly:
(env)user/om/user/user/MEng/hbf_tensorflow_code/tf_experiments_scripts $ python ns_test.py
['README', 'my-tf-proj.egg-link', 'tensorflow-0.9.0.dist-info', 'wheel', 'Keras-1.0.6.dist-info', 'numpy-1.11.1-py2.7.egg-info', 'wheel-0.29.0.dist-info', 'easy-install.pth', 'PyYAML-3.11.dist-info', 'sklearn', 'easy_install.pyc', 'google', 'tensorflow', 'namespaces', 'scikit_learn-0.17.1-py2.7.egg-info', 'setuptools-24.0.0.dist-info', 'wheel-0.29.0-py2.7.egg-info', 'keras', 'Theano-0.8.2.dist-info', 'easy_install.py', 'six.pyc', 'yaml', 'protobuf-3.0.0b2-py2.7-nspkg.pth', 'numpy-1.11.1.dist-info', 'namespaces-1.0.0.dist-info', 'pip-8.1.2-py2.7.egg-info', 'setuptools', 'theano', 'six-1.10.0.dist-info', 'setuptools-23.0.0-py2.7.egg', 'pip', 'setuptools.pth', 'six.py', 'protobuf-3.0.0b2.dist-info', 'scipy-0.17.1-py2.7.egg-info', 'numpy', 'external', 'pkg_resources', 'scipy']
True
<module 'namespaces' from '/home/user/envs/tensorflow/lib/python2.7/site-packages/namespaces/__init__.pyc'>
when I do srun python ns_test.py it again prints the above. However, when I use sbatch it prints (to the slurm log) the following mysterious lines of code:
['README', 'my-tf-proj.egg-link', 'tensorflow-0.9.0.dist-info', 'wheel', 'Keras-1.0.6.dist-info', 'numpy-1.11.1-py2.7.egg-info', 'wheel-0.29.0.dist-info', 'easy-install.pth', 'PyYAML-3.11.dist-info', 'sklearn', 'easy_install.pyc', 'google', 'tensorflow', 'namespaces', 'scikit_learn-0.17.1-py2.7.egg-info', 'setuptools-24.0.0.dist-info', 'wheel-0.29.0-py2.7.egg-info', 'keras', 'Theano-0.8.2.dist-info', 'easy_install.py', 'six.pyc', 'yaml', 'protobuf-3.0.0b2-py2.7-nspkg.pth', 'numpy-1.11.1.dist-info', 'namespaces-1.0.0.dist-info', 'pip-8.1.2-py2.7.egg-info', 'setuptools', 'theano', 'six-1.10.0.dist-info', 'setuptools-23.0.0-py2.7.egg', 'pip', 'setuptools.pth', 'six.py', 'protobuf-3.0.0b2.dist-info', 'scipy-0.17.1-py2.7.egg-info', 'numpy', 'external', 'pkg_resources', 'scipy']
True
Traceback (most recent call last):
file "/home/slurm/slurmd/job3210331/slurm_script", line 12, in <module>
import namespaces as ns
ImportError: No module named namespaces
which is really mysterious because it does say that the library is present there! However, when I use the import statement with sbatch it doesn't find it. Why might it be? How I can I fix this?
Most likely issue is that your slurm isn't using the right python interpeter and therefore right python sys.path. To fix that tell slurm at the top of the submission script what type of file it is by pointing to the right python intepreter e.g. to use an environment python:
#!/usr/bin/env python
this one points to the system default one:
#!/bin/python
for another example using a conda env:
#!/home/miranda9/miniconda3/envs/automl-meta-learning/bin/python
if you have a conda env activated you can find the path with which python termianl command e.g.:
(iit_term_synthesis) brandomiranda~ ❯ which python
/Users/brandomiranda/miniconda/envs/iit_term_synthesis/bin/python