Search code examples
pandasjupyter-notebookcondaenvironment

Conda and Jupyter Notebook Environment Confusion


I am using Jupyter Notebook to help debug some issues I'm having moving between JSON and pandas. The specific application isn't important.

The important part is that I needed to use pandas.json_normalize() which apparently first showed up in pandas version 1.0.3. I was confused when Jupyter said it doesn't exist. I did a version check and got:

In[]: pd.__version
Out[]: 0.25.2

This is not the version of python installed in either my base environment or the conda environment that Jupyter Notebook is running in or that the app is running in. Version checks in both environments in Anaconda Prompt (outside of Jupyter Notebook) confirm this.

What is going on here? Looking around I haven't seen a good answer, but it does appear that other people have had the same issue --- Jupyter defaulting to pandas 0.25.2 for some reason.


Solution

  • It seems that your Notebook is using a different kernel/environment than what you want.
    run this in the notebook to see which environment you are using

    ! which python
    

    or try

    import sys
    print(sys.executable)
    

    which would show you which environment it's using, if you have env named venv then you will get something like.

    /home/your_home_directory/anaconda3/envs/venv/bin/python
    

    If you don't care about all of that and you just want to update the pandas that it's using then copy that path and do this.

    ! pip install --upgrade pandas
    

    Note that this will also depend on which version of python you are using