ImportError: cannot import name 'Iterator' from 'typing_extensions' (/databricks/python/lib/python3.10/site-packages/typing_extensions.py)

Question

I'm facing the following issue on Azure Databricks when trying to import from openai import OpenAI after installing openai.

Here's the error:

ImportError: cannot import name 'Iterator' from 'typing_extensions' (/databricks/python/lib/python3.10/site-packages/typing_extensions.py)

I looked up similar issues and found that using --force-reinstall like this:

pip install --force-reinstall typing-extensions==4.5
pip install --force-reinstall openai==1.8

Worked for some users. However, it did not work in my case.

How do I resolve this?

Answer 1

To resolve this erratic issue simply run dbutils.library.restartPython() after installing openai and before importing from openai import OpenAI like this

!pip install openai==1.42.0

# Restart the Python process on Databricks
dbutils.library.restartPython()

from openai import OpenAI

As the Databricks Documentation says:

You can programmatically restart the Python process on Databricks (using dbutils.library.restartPython()) to ensure that locally installed or upgraded libraries function correctly in the Python kernel for your current SparkSession.