How to use the Python openai client with both Azure and OpenAI at the same time?

Question

OpenAI offers a Python client, currently in version 0.27.8, which supports both Azure and OpenAI.

Here are examples of how to use it to call the ChatCompletion for each provider:

# openai_chatcompletion.py

"""Test OpenAI's ChatCompletion endpoint"""
import os
import openai
import dotenv
dotenv.load_dotenv()
openai.api_key = os.environ.get('OPENAI_API_KEY')

# Hello, world.
api_response = openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hello!"}
  ],
  max_tokens=16,
  temperature=0,
  top_p=1,
  frequency_penalty=0,
  presence_penalty=0,
)

print('api_response:', type(api_response), api_response)
print('api_response.choices[0].message:', type(api_response.choices[0].message), api_response.choices[0].message)

And:

# azure_openai_35turbo.py

"""Test Microsoft Azure's ChatCompletion endpoint"""
import os
import openai
import dotenv
dotenv.load_dotenv()

openai.api_type = "azure"
openai.api_base = os.getenv("AZURE_OPENAI_ENDPOINT") 
openai.api_version = "2023-05-15"
openai.api_key = os.getenv("AZURE_OPENAI_KEY")


# Hello, world.
# In addition to the `api_*` properties above, mind the difference in arguments
# as well between OpenAI and Azure:
# - OpenAI from OpenAI uses `model="gpt-3.5-turbo"`!
# - OpenAI from Azure uses `engine="‹deployment name›"`! ⚠️
#   > You need to set the engine variable to the deployment name you chose when
#   > you deployed the GPT-35-Turbo or GPT-4 models.
#  This is the name of the deployment I created in the Azure portal on the resource.
api_response = openai.ChatCompletion.create(
  engine="gpt-35-turbo", # engine = "deployment_name".
  messages=[
    {"role": "user", "content": "Hello!"}
  ],
  max_tokens=16,
  temperature=0,
  top_p=1,
  frequency_penalty=0,
  presence_penalty=0,
)

print('api_response:', type(api_response), api_response)
print('api_response.choices[0].message:', type(api_response.choices[0].message), api_response.choices[0].message)

i.e. api_type and other settings are globals of the Python library.

Here is a third example to transcribe audio (it uses Whisper, which is available on OpenAI but not on Azure):

# openai_transcribe.py

"""
Test the transcription endpoint
https://platform.openai.com/docs/api-reference/audio
"""
import os
import openai
import dotenv
dotenv.load_dotenv()


openai.api_key = os.getenv("OPENAI_API_KEY")
audio_file = open("minitests/minitests_data/bilingual-english-bosnian.wav", "rb")
transcript = openai.Audio.transcribe(
    model="whisper-1",
    file=audio_file,
    prompt="Part of a Bosnian language class.",
    response_format="verbose_json",
)
print(transcript)

These are minimal examples but I use similar code as part of my webapp (a Flask app).

Now my challenge is that I'd like to:

1. Use the ChatCompletion endpoint from Azure; but:
2. Use the Transcribe endpoint from OpenAI (since it's not available on Azure)

Is there any way to do so?

I have a few options in mind:

• Changing the globals before every call. But I'm worried that this might cause side-effects I did not expect.
• Duplicating/Forking the library to have two versions run concurrently, one for each provider, but this also feels very messy.
• Use an alternative client for OpenAI's Whisper, if any.

I'm not too comfortable with these and feel I may have missed a more obvious solution.

Or of course… Alternatively, I could just use Whisper with a different provider (e.g. Replicate) or an alternative to Whisper altogether.

See also

• Someone reported the issue (but without a solution) on GitHub (openai/openai-python): Using Azure and OpenAI at the same time #411

Answer 1

Each API in the library accepts per-method overrides for the configuration options. If you want to access the Azure API for chat completions, you can explicitly pass in your Azure config. For the transcribe endpoint, you can explicitly pass the OpenAI config. For example:

import os
import openai

api_response = openai.ChatCompletion.create(
    api_base=os.getenv("AZURE_OPENAI_ENDPOINT"),
    api_key=os.getenv("AZURE_OPENAI_KEY"),
    api_type="azure",
    api_version="2023-05-15",
    engine="gpt-35-turbo",
    messages=[
    {"role": "user", "content": "Hello!"}
    ],
    max_tokens=16,
    temperature=0,
    top_p=1,
    frequency_penalty=0,
    presence_penalty=0,
)
print(api_response)



audio_file = open("minitests/minitests_data/bilingual-english-bosnian.wav", "rb")
transcript = openai.Audio.transcribe(
    api_key=os.getenv("OPENAI_API_KEY"),
    model="whisper-1",
    file=audio_file,
    prompt="Part of a Bosnian language class.",
    response_format="verbose_json",
)
print(transcript)