Search code examples
azureazure-openai

How do I make a secure connection from Azure OpenAI to an Azure Search instance?

I've built out an Azure OpenAI solution using RAG on my own data via an Azure AI Search index.

So I've added my Azure AI Search instance as a data source in Azure OpenAI. But I can only get it to connect to and use my search index if enable public access on my AI Search instance.

How do I create a secure network between Azure OpenAI and Azure AI Search, so that I don't have to enable public access on AI Search?

I've tried adding OpenAI to my Vnet. I added a private endpoint connection in AI Search. Then I added a shared private access in AI search which I approved from OpenAI. But like this link shows, I think that's for communication outbound from AI Search. Or inbound to OpenAI.

Or this one is more about security at the user level. Or this one is about secure admin access to AI Search.

I'm new to Azure and using vnets and obviously missing something. Any help would be much appreciated. Thanks!

Solution

  • You got a good point and find out the right documentation (mentioned in your comment: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/use-your-data-securely#disable-public-network-access-1)

    There are several points:

    Generally speaking, when doing "proper security and networking" in Azure, you should:

    • use RBAC instead of service keys
    • disable public access on resources
    • add private endpoints to access the service

    This should be enough.

    But... in this "Azure OpenAI on your data", if you check the API call made (for example from the Azure OpenAI Studio), Microsoft has created an 'all-in-one' endpoint in its OpenAI API where you define a 'dataSource' in the POST body, pointing to your Azure AI Search service and providing the API key of this service! So it's not really you who are calling your Azure AI Search, but it's made from a Microsoft-owned network... That's why you have to fill a form, so that they "will provision the private endpoint in Microsoft managed virtual network, and send a private endpoint connection request to your search service". It makes an important difference in terms of security...

    What I would do to avoid that and be 100% responsible of the network and security aspects:

    • on my Azure AI Search and Azure OpenAI resources:
      • disable public access
      • add private endpoints
    • make my own API (using an App Service for example) which will be used by my consumer UI/UX (webchat or other)
      • activate its system identity
      • add permissions to this identity on my Azure AI Search and Azure OpenAI resources (using RBAC)
      • add VNet integration (in the same VNet where the services private endpoints are set) and force the output traffic to go through the VNet
      • and split the call to Azure OpenAI to avoid using this "extension" where the Search details are provided:
        • make the call to Azure AI Search first
        • and call Azure OpenAI with my own prompt containing the search results (and it will also avoid having issues with "not working system prompt" when doing the "all-in-one" call)