I've set up a GPT4 deployment and a chat playground. The chat playground makes use of the add your data feature, using Azure Cognitive Search semantic search. I've also provided a non-default system message. Responses are limited to my data content.
When I send messages through the chat playground, it gives me responses as I would expect, drawing on the references that I ingested. However, when I use the code I find in "view code" in the chat session window and send the exact same user message, I get a response containing the message "The requested information is not available in the retrieved data. Please try another query or topic."
Here is my code:
import openai, os, requests
openai.api_type = "azure"
# Azure OpenAI on your own data is only supported by the 2023-08-01-preview API version
openai.api_version = "2023-08-01-preview"
# Azure OpenAI setup
openai.api_base = BASE_URL
openai.api_key = OPENAI_API_KEY
deployment_id = DEPLOYMENT_NAME
# Azure Cognitive Search setup
search_endpoint = COGNITIVE_SEARCH_ENDPOINT
search_key = SEARCH_KEY
search_index_name = SEARCH_INDEX_NAME
def setup_byod(deployment_id: str) -> None:
"""Sets up the OpenAI Python SDK to use your own data for the chat endpoint.
:param deployment_id: The deployment ID for the model to use with your own data.
To remove this configuration, simply set openai.requestssession to None.
"""
class BringYourOwnDataAdapter(requests.adapters.HTTPAdapter):
def send(self, request, **kwargs):
request.url = f"{openai.api_base}/openai/deployments/{deployment_id}/extensions/chat/completions?api-version={openai.api_version}"
return super().send(request, **kwargs)
session = requests.Session()
# Mount a custom adapter which will use the extensions endpoint for any call using the given `deployment_id`
session.mount(
prefix=f"{openai.api_base}/openai/deployments/{deployment_id}",
adapter=BringYourOwnDataAdapter()
)
openai.requestssession = session
setup_byod(deployment_id)
completion = openai.ChatCompletion.create(
messages=[{"role": "system", "content": "<SYSTEM MESSAGE>"},
{"role": "user", "content": "<QUERY"}],
deployment_id=deployment_id,
dataSources=[
{
"type": "AzureCognitiveSearch",
"parameters": {
"endpoint": search_endpoint,
"key": search_key,
"indexName": search_index_name,
}
}
]
)
print(completion)
The code initially provided in the "view code" window did not have the system prompt I was using in the playground, so I added that, but it didn't solve the problem. I've double and triple checked all the values under # Azure OpenAI setup and # Azure Cognitive Search setup so it's not that I'm passing the wrong values there. What could be causing this discrepancy in the behavior?
The “view code” should add disclaimer that it is not supposed to generate the same result as playground, because it only populates the API required parameters, and doesn't populate any optional parameters, which also affect the retrieval quality. For example, inScope, strictness...Note the system message from UX is send as dataSources[0].parameters.roleInformation field, not in messages[0].
The best way to repro the playground result is to use F12 to capture the network request send to your AOAI /extensions/chat/completions API, and then click "Save as cURL (bash)". Then map the parameters to your code to call SDK.