Getting 404 on Openai Azure Endpoint

Question

Using the AzureOpenAi client, and getting a 404

AsyncAzureOpenAI(
                api_key=os.getenv("AZURE_OPENAI_API_KEY"),  
                api_version="2024-01-25-preview",
                azure_deployment="XXX-staging",
                azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT", ""),
            )

As you can see in the image below, I'm using the 0125 model version. Interestingly, I don't even see my model version listed directly here: https://learn.microsoft.com/en-us/azure/ai-services/openai/api-version-deprecation

I've tried this on openai-1.23.2 and openai-1.14.3

Answer 1

Found the answer.

Found the answer. The problem was actually in the request params.Normally, openai expects a model name to be passed in for model like

gpt-4-1106-preview

But in the case of azure, it expects the given model name in your deployment, so whatever name you wrote down for the deployment should be passed in here.

params = {
    "model": model.name, // NAME IN AZURE
    "messages": chat_messages,
    "max_tokens": model_max_tokens,
    "temperature": temperature,
    "stream": stream,
}