python jupyter-notebook kernel python-3.6 jupyter-lab

Jupyter Lab and Notebook Problem: Kernel Error

It appears that somehow one of my kernels is deleted. And perhaps that's why I am getting this error. How could I possibly fix it? Can I reinstall something (the kernel in the base environment perhaps)?

If no better option is available, I would try reinstalling anaconda altogether as a last resort according to: Default kernel in jupyter notebook (Python3) not working.

I have tried using the following:

Current List of Kernels

jupyter kernelspec list

Output:

Available kernels:
  python3    C:\Users\raysu\AppData\Roaming\jupyter\kernels\python3
  vpython    C:\ProgramData\jupyter\kernels\vpython

But I must have another kernel (which I suppose somehow got deleted) associated with my base environment.

C:\Users\raysu\Documents>conda env list
# conda environments:
#
base                  *  C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64
astra_env                C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\envs\astra_env
bokeh_env                C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\envs\bokeh_env
sound_env                C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\envs\sound_env
tf_env                   C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\envs\tf_env
                         C:\Users\raysu\Anaconda3
                         C:\Users\raysu\Anaconda3\envs\pyforge
opencv_env               C:\Users\raysu\AppData\Local\conda\conda\envs\opencv_env
test_env                 C:\Users\raysu\AppData\Local\conda\conda\envs\test_env

Error

Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\tornado\web.py", line 1699, in _execute
    result = await result
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\tornado\gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\notebook\services\sessions\handlers.py", line 72, in post
    type=mtype))
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\tornado\gen.py", line 735, in run
    value = future.result()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\tornado\gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 88, in create_session
    kernel_id = yield self.start_kernel_for_session(session_id, path, name, type, kernel_name)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\tornado\gen.py", line 735, in run
    value = future.result()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\tornado\gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 101, in start_kernel_for_session
    self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\tornado\gen.py", line 735, in run
    value = future.result()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\tornado\gen.py", line 209, in wrapper
    yielded = next(result)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\notebook\services\kernels\kernelmanager.py", line 168, in start_kernel
    super(MappingKernelManager, self).start_kernel(**kwargs)
  File "C:\Users\raysu\AppData\Roaming\Python\Python36\site-packages\jupyter_client\multikernelmanager.py", line 110, in start_kernel
    km.start_kernel(**kwargs)
  File "C:\Users\raysu\AppData\Roaming\Python\Python36\site-packages\jupyter_client\manager.py", line 240, in start_kernel
    self.write_connection_file()
  File "C:\Users\raysu\AppData\Roaming\Python\Python36\site-packages\jupyter_client\connect.py", line 476, in write_connection_file
    kernel_name=self.kernel_name
  File "C:\Users\raysu\AppData\Roaming\Python\Python36\site-packages\jupyter_client\connect.py", line 141, in write_connection_file
    with secure_write(fname) as f:
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "C:\Users\raysu\AppData\Roaming\Python\Python36\site-packages\jupyter_core\paths.py", line 404, in secure_write
    win32_restrict_file_to_user(fname)
  File "C:\Users\raysu\AppData\Roaming\Python\Python36\site-packages\jupyter_core\paths.py", line 359, in win32_restrict_file_to_user
    import win32api
ImportError: DLL load failed: The specified procedure could not be found.

Solution

Solved the problem by editing the relevant kernelspec: kernel.json file after fresh installation of Anaconda.

Solution in Detail

As reported earlier, I found that I had two kernelspecs. (command: jupyter kernelspec list)

python3: C:\Users\raysu\AppData\Roaming\jupyter\kernels\python3
vpython: C:\ProgramData\jupyter\kernels\vpython

The kernelspec is a .json file that maintains kernel-specifications for jupyter-notebook/lab to launch a certain kernel. Fortunately, the kernel vpython was functioning and that helped me narrow down the source of the problem located inside kernelpsec: python3. I had to correct the path to the python executable (python.exe) and that solved the issue.

I would like to mention that in this case the following two resources came handy. I got more clarity in where the problem could be, after I saw the typical-structure of the kernelspec document.

Note: However, since I cannot test this theory, I would ask anyone else in a similar problem, to first make a copy of the old kernel.json file and then update the path to python.exe file. If that resolves the kernel error, then reinstalling Anaconda altogether may not be necessary.

Long story short:

If your jupyter notebook/lab environment shows you kernel error, most likely the source of the problem is inside the kernelspec file: kernel.json and the path to python.exe is incorrect. This could be a more likely scenario, if the same kernel error persists even after fresh re-installation of anaconda/jupyter-notebook/lab.

For instance, in my case, the python3-kernelspec was located at:

"C:\Users\raysu\AppData\Roaming\jupyter\kernels\python3"

And the python.exe was located at:

"C:\Users\raysu\Anaconda3\python.exe"

However, the path to the python-executable was wrong: it was still showing the path from the old installation. And post-anaconda-reinstallation, this path had changed. So, I had to update the kernelspec for python3 by editing kernel.json file.

# kernel.json file
{
 "argv": [
  "C:\\Users\\raysu\\Anaconda3\\python.exe",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3",
 "language": "python"
}

If this does not solve your problem, please consult the following two documents. I wasted 4 days because of this. Hopefully, this answer would help out someone in future.

Jupyter Lab and Notebook Problem: Kernel Error

Current List of Kernels

Error

Solution in Detail

Long story short:

What came handy?