Search code examples
azureazure-cognitive-searchazure-openai

Azure Open AI failing to read Search Index?

Migrating Question Answering to Open AI for chat bot. Source data is few thousand 3k json files look like the below in Blob Storage, Azure AI Search data/indexer/index is setup. Index is set using Complex Type. Azure Open AI GUI does not list any of the fields in the complex type in the Content data, file name, title, or url field dropdowns. What is the proper way to create an index that AI will recognize the Complex Data fields?

Thanks.

{
"id": 303,
"answer": "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua",
"questions": [
    "Onteger vitae justo eget magna fermentum iaculis eu non diam" , 
    "Faucibus interdum posuere lorem ipsum dolor sit. Integer feugiat scelerisque varius morbi enim nunc faucibus a"
],
"mediaUris": [
    "uri1.nu" , 
    "uri2.nu"
],
"sourceRefUris": [
    "uria.nu" , 
    "urib.nu"
],
"questionsRelated": null

}

Search Index json definition is https://pastebin.com/sBafB2LD

AI Chat Playground gives error "Azure Search: No content fields found. Please confirm if you have String fields which are searchable!" when attempting to use the built-in chat functionality.

Solution

  • According to this documentation, the .json format is not supported. Additionally, OpenAI does not display complex and collection types when adding your own data.

    To address this issue, you can set your index fields as string type by following these steps:

    1. In Azure AI Search, click Import data and select the data source.

    enter image description here

    1. Next, click on Customize target index.

    enter image description here

    1. In the index settings, remove all fields except metadata_storage_path, metadata_storage_name, and metadata_storage_last_modified. Set metadata_storage_last_modified as the key.

    enter image description here

    1. Add your fields one by one, setting them all as string type.

    enter image description here

    1. Click on "Create indexer" and go to Advanced options. Select Parsing mode as JSON.

    enter image description here

    After clicking on submit, you will see an index with all fields set as string type.

    enter image description here

    Now, you can select fields in Azure OpenAI.

    enter image description here

    and

    enter image description here