pythontensorflowgpu
TensorFlow GPU recognized in the terminal but not in the Jupyter notebook
I installed TensorFlow with the GPU support according to the official installation page and GPU is recognized from the terminal but not from the Jupyter notebook with the Jupyter kernel from the same Conda environment tensor_gpu (see the screenshot below). The Jupyter Lab 3.6.3 (Windows 10) is installed and run from the separate Conda environment jupyter.
I also see the following warning in the Jupyter Lab console:
2023-04-25 16:34:44.493879: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2023-04-25 16:34:44.494185: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2023-04-25 16:34:47.012660: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2023-04-25 16:34:47.014859: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cublas64_11.dll'; dlerror: cublas64_11.dll not found
2023-04-25 16:34:47.017536: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cublasLt64_11.dll'; dlerror: cublasLt64_11.dll not found
2023-04-25 16:34:47.019433: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cufft64_10.dll'; dlerror: cufft64_10.dll not found
2023-04-25 16:34:47.021223: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'curand64_10.dll'; dlerror: curand64_10.dll not found
2023-04-25 16:34:47.023012: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cusolver64_11.dll'; dlerror: cusolver64_11.dll not found
2023-04-25 16:34:47.024836: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cusparse64_11.dll'; dlerror: cusparse64_11.dll not found
2023-04-25 16:34:47.026672: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudnn64_8.dll'; dlerror: cudnn64_8.dll not found
2023-04-25 16:34:47.026872: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
Could you please suggest what I am missing here, like environment variables, additional paths, etc.?
The list of installed packages in the environment tensor_gpu:
(tensor_gpu) C:\Users\Pavlo Fesenko\Desktop>mamba list
# packages in environment at C:\Users\Pavlo Fesenko\.conda\envs\tensor_gpu:
#
# Name Version Build Channel
absl-py 1.4.0 pypi_0 pypi
asttokens 2.2.1 pyhd8ed1ab_0 conda-forge
astunparse 1.6.3 pypi_0 pypi
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 pyhd8ed1ab_3 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
brotli 1.0.9 hcfcfb64_8 conda-forge
brotli-bin 1.0.9 hcfcfb64_8 conda-forge
brotlipy 0.7.0 py39ha55989b_1005 conda-forge
bzip2 1.0.8 h8ffe710_4 conda-forge
ca-certificates 2022.12.7 h5b45459_0 conda-forge
cachetools 5.3.0 pypi_0 pypi
certifi 2022.12.7 pypi_0 pypi
cffi 1.15.1 py39h68f70e3_3 conda-forge
charset-normalizer 3.1.0 pyhd8ed1ab_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
comm 0.1.3 pyhd8ed1ab_0 conda-forge
contourpy 1.0.7 py39h1f6ef14_0 conda-forge
cryptography 40.0.2 py39hb6bd5e6_0 conda-forge
cudatoolkit 11.2.2 h7d7167e_11 conda-forge
cudnn 8.1.0.77 h3e0f4f4_0 conda-forge
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
debugpy 1.6.7 py39h99910a6_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
executing 1.2.0 pyhd8ed1ab_0 conda-forge
flatbuffers 23.3.3 pypi_0 pypi
fonttools 4.39.3 py39ha55989b_0 conda-forge
freetype 2.12.1 h546665d_1 conda-forge
gast 0.4.0 pypi_0 pypi
gettext 0.21.1 h5728263_0 conda-forge
glib 2.76.2 h12be248_0 conda-forge
glib-tools 2.76.2 h12be248_0 conda-forge
google-auth 2.17.3 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
grpcio 1.54.0 pypi_0 pypi
gst-plugins-base 1.22.0 h001b923_2 conda-forge
gstreamer 1.22.0 h6b5321d_2 conda-forge
h5py 3.8.0 pypi_0 pypi
icu 72.1 h63175ca_0 conda-forge
idna 3.4 pyhd8ed1ab_0 conda-forge
importlib-metadata 6.6.0 pyha770c72_0 conda-forge
importlib-resources 5.12.0 pyhd8ed1ab_0 conda-forge
importlib_metadata 6.6.0 hd8ed1ab_0 conda-forge
importlib_resources 5.12.0 pyhd8ed1ab_0 conda-forge
intel-openmp 2023.1.0 h57928b3_46319 conda-forge
ipykernel 6.22.0 pyh025b116_0 conda-forge
ipython 8.12.0 pyh08f2357_0 conda-forge
jedi 0.18.2 pyhd8ed1ab_0 conda-forge
jupyter_client 8.2.0 pyhd8ed1ab_0 conda-forge
jupyter_core 5.3.0 py39hcbf5309_0 conda-forge
keras 2.10.0 pypi_0 pypi
keras-preprocessing 1.1.2 pypi_0 pypi
kiwisolver 1.4.4 py39h1f6ef14_1 conda-forge
krb5 1.20.1 heb0366b_0 conda-forge
lcms2 2.15 h3e3b177_1 conda-forge
lerc 4.0.0 h63175ca_0 conda-forge
libblas 3.9.0 16_win64_mkl conda-forge
libbrotlicommon 1.0.9 hcfcfb64_8 conda-forge
libbrotlidec 1.0.9 hcfcfb64_8 conda-forge
libbrotlienc 1.0.9 hcfcfb64_8 conda-forge
libcblas 3.9.0 16_win64_mkl conda-forge
libclang 16.0.0 pypi_0 pypi
libclang13 16.0.2 default_h45d3cf4_0 conda-forge
libdeflate 1.18 hcfcfb64_0 conda-forge
libffi 3.4.2 h8ffe710_5 conda-forge
libglib 2.76.2 he8f3873_0 conda-forge
libhwloc 2.9.1 h51c2c0f_0 conda-forge
libiconv 1.17 h8ffe710_0 conda-forge
libjpeg-turbo 2.1.5.1 hcfcfb64_0 conda-forge
liblapack 3.9.0 16_win64_mkl conda-forge
libogg 1.3.4 h8ffe710_1 conda-forge
libpng 1.6.39 h19919ed_0 conda-forge
libsodium 1.0.18 h8d14728_1 conda-forge
libsqlite 3.40.0 hcfcfb64_1 conda-forge
libtiff 4.5.0 h6c8260b_6 conda-forge
libvorbis 1.3.7 h0e60522_0 conda-forge
libwebp-base 1.3.0 hcfcfb64_0 conda-forge
libxcb 1.13 hcd874cb_1004 conda-forge
libxml2 2.10.4 hc3477c8_0 conda-forge
libzlib 1.2.13 hcfcfb64_4 conda-forge
m2w64-gcc-libgfortran 5.3.0 6 conda-forge
m2w64-gcc-libs 5.3.0 7 conda-forge
m2w64-gcc-libs-core 5.3.0 7 conda-forge
m2w64-gmp 6.1.0 2 conda-forge
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge
markdown 3.4.3 pypi_0 pypi
markupsafe 2.1.2 pypi_0 pypi
matplotlib 3.7.1 py39hcbf5309_0 conda-forge
matplotlib-base 3.7.1 py39haf65ace_0 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mkl 2022.1.0 h6a75c08_874 conda-forge
msys2-conda-epoch 20160418 1 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge
numpy 1.24.3 py39h816b6a6_0 conda-forge
oauthlib 3.2.2 pypi_0 pypi
openjpeg 2.5.0 ha2aaf27_2 conda-forge
openssl 3.1.0 hcfcfb64_2 conda-forge
opt-einsum 3.3.0 pypi_0 pypi
packaging 23.1 pyhd8ed1ab_0 conda-forge
pandas 2.0.1 py39h1679cfb_0 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
pcre2 10.40 h17e33f8_0 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 9.5.0 py39haa1d754_0 conda-forge
pip 23.1.1 pyhd8ed1ab_0 conda-forge
platformdirs 3.2.0 pyhd8ed1ab_0 conda-forge
ply 3.11 py_1 conda-forge
pooch 1.7.0 pyha770c72_3 conda-forge
prompt-toolkit 3.0.38 pyha770c72_0 conda-forge
prompt_toolkit 3.0.38 hd8ed1ab_0 conda-forge
protobuf 3.19.6 pypi_0 pypi
psutil 5.9.5 py39ha55989b_0 conda-forge
pthread-stubs 0.4 hcd874cb_1001 conda-forge
pthreads-win32 2.9.1 hfa6e2cd_3 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pyasn1 0.5.0 pypi_0 pypi
pyasn1-modules 0.3.0 pypi_0 pypi
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.15.1 pyhd8ed1ab_0 conda-forge
pyopenssl 23.1.1 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pyqt 5.15.7 py39hb77abff_3 conda-forge
pyqt5-sip 12.11.0 py39h99910a6_3 conda-forge
pysocks 1.7.1 pyh0701188_6 conda-forge
python 3.9.16 h4de0772_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge
python_abi 3.9 3_cp39 conda-forge
pytz 2023.3 pyhd8ed1ab_0 conda-forge
pywin32 304 py39h99910a6_2 conda-forge
pyzmq 25.0.2 py39hea35a22_0 conda-forge
qt-main 5.15.8 h7f2b912_9 conda-forge
requests 2.28.2 pyhd8ed1ab_1 conda-forge
requests-oauthlib 1.3.1 pypi_0 pypi
rsa 4.9 pypi_0 pypi
scipy 1.10.1 py39hde5eda1_0 conda-forge
setuptools 67.7.2 pyhd8ed1ab_0 conda-forge
sip 6.7.9 py39h99910a6_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
tbb 2021.9.0 h91493d7_0 conda-forge
tensorboard 2.10.1 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
tensorflow 2.10.1 pypi_0 pypi
tensorflow-estimator 2.10.0 pypi_0 pypi
tensorflow-io-gcs-filesystem 0.31.0 pypi_0 pypi
termcolor 2.3.0 pypi_0 pypi
tk 8.6.12 h8ffe710_0 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
tornado 6.3 py39ha55989b_0 conda-forge
traitlets 5.9.0 pyhd8ed1ab_0 conda-forge
typing-extensions 4.5.0 hd8ed1ab_0 conda-forge
typing_extensions 4.5.0 pyha770c72_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
ucrt 10.0.22621.0 h57928b3_0 conda-forge
unicodedata2 15.0.0 py39ha55989b_0 conda-forge
urllib3 1.26.15 pyhd8ed1ab_0 conda-forge
vc 14.3 hb6edc58_10 conda-forge
vs2015_runtime 14.34.31931 h4c5c07a_10 conda-forge
wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge
werkzeug 2.2.3 pypi_0 pypi
wheel 0.40.0 pyhd8ed1ab_0 conda-forge
win_inet_pton 1.1.0 pyhd8ed1ab_6 conda-forge
wrapt 1.15.0 pypi_0 pypi
xorg-libxau 1.0.9 hcd874cb_0 conda-forge
xorg-libxdmcp 1.1.3 hcd874cb_0 conda-forge
xz 5.2.6 h8d14728_0 conda-forge
zeromq 4.3.4 h0e60522_1 conda-forge
zipp 3.15.0 pyhd8ed1ab_0 conda-forge
zstd 1.5.2 h12be248_6 conda-forge
Solution
In the pytorch environment, the python file normally calls the GPU, but jupyter cannot call the GPU
Solution:
The first step is to view the previous kernel location
jupyter kernelspec list
The second step is to delete the previous kernel
jupyter kernelspec remove 'your kernel name'
I am here jupyter kernelspec remove python3
The third step is to install the nb_conda_kernels in base environment package to automatically generate the kernel from python in the virtual environment
conda install nb_conda_kernels
Wait for the installation to complete
The test successfully calls cuda