Azure AI Studio - ML Index GPT4 deployment API calls not working due to access_token error

Question

I have built an Azure AI GPT4 model, and I have begun using it normally via API calls and all has worked fine. I have recently, created an Azure Search and ML Index to give it some custom data in the chat. This has all worked well and works in the Playground of Azure AI Studio.

However, when I have used the JSON code provided to setup my API, I am receiving the below return from the API:

{
  "error": {
    "requestid": "<request_id>",
    "code": 400,
    "message": "Failed to access workspace /subscriptions/<subscription_guid>/resourceGroups/<resource_group_name>/providers/Microsoft.MachineLearningServices/workspaces/<workspace_name> due to invalid token. Response: {\"error\":{\"code\":\"InvalidAuthenticationToken\",\"message\":\"The access token is invalid.\"}}"
  }
}

The code provided by Microsoft in the Azure AI Studio playground, tells you to use the access_token authentication, but doesn't actually supply the value, or parameters. I found out the syntax in this documentation, and added what I thought, was the supplied access_token, but it always fails.

I have searched through all documentation I can find, and through Azure Portal, and I cannot find where an access token would be for me to include.

If I setup the API to use AI Search instead of Azure ML Index, it works, but doesn't use the ML index, which isn't right.

How can I get this to work correctly, or where can I find or generate an access token?

Full (obfuscated) code for my API call below:

{
  "data_sources": [
    {
      "type": "azure_ml_index",
      "parameters": {
        "project_resource_id": "/subscriptions/<subscription_guid>/resourceGroups/<resource_group_name>/providers/Microsoft.MachineLearningServices/workspaces/<workspace_name>",
        "name": "<index name>",
        "version": "1",
        "authentication": {
          "type": "access_token",
          "access_token": "<UNKNOWN>"
        },
        "query_type": "vector",
        "in_scope": true,
        "role_information": "<system prompt>",
        "strictness": 3,
        "top_n_documents": 5,
        "endpoint": "<azure search endpoint>",
        "key": "<azure search key>",
        "indexName": "<gpt4 index name>"
      }
    }
  ],
  "messages": [
    {
      "role": "system",
      "content": "<system prompt>"
    },
    {
      "role": "user",
      "content": "<content prompt>"
    },
    {
      "role": "assistant",
      "content": "<response prompt>"
    }
  ],
  "deployment": "<gpt4model>",
  "temperature": 0.5,
  "top_p": 1,
  "max_tokens": 1800,
  "stop": null,
  "stream": true,
  "frequency_penalty": 0.25,
  "presence_penalty": 0.35,
  "azureSearchEndpoint": "<azure search endpoint>",
  "azureSearchKey": "<azure search key>"
}

Answer 1

You need to have an access token for accessing resources inside the machine learning workspace.

To get the token and use it in a POST request, use the Azure CLI commands below:

token_response=$(az account get-access-token --resource=https://management.core.windows.net/)

access_token=$(echo "$token_response" | jq -r '.accessToken')

az rest --method POST --uri https://jgsopenai.openai.azure.com/openai/deployments/tst/chat/completions?api-version=2024-02-15-preview --resource https://cognitiveservices.azure.com/ --body '
{
    "data_sources": [
      {
        "type": "azure_ml_index",
        "parameters": {
          "project_resource_id": "/subscriptions/b83c1ed3-xxxxxxxxxx/resourceGroups/<xxx-yyy>/providers/Microsoft.MachineLearningServices/workspaces/<ml-workspace-name>",
          "name": "maroon-gyro-lw5ss1dmjt",
          "version": "1",
          "authentication": {
          "type": "access_token",
          "access_token": "'"$access_token"'"
        }
        }
      }
    ],
    "messages": [
      {
        "role": "user",
        "content": "What is the amount due paid for the current bill?"
      }
    ]
}
'

Here, I am getting the access token first and providing it in the POST request. Also, make sure you have done az login in the bash CLI.

Output:

You can also get the token using a service principal.

Below are the commands to get token using service principal.

token_response=$(curl -X POST https://login.microsoftonline.com/<tenent_id>/oauth2/token -d "grant_type=client_credentials&resource=https%3A%2F%2Fmanagement.azure.com%2F&client_id=<client_id>&client_secret=<client_seceret>")

access_token=$(echo "$token_response" | jq -r '.access_token')


az rest --method POST --uri https://jgsopenai.openai.azure.com/openai/deployments/tst/chat/completions?api-version=2024-02-15-preview --resource https://cognitiveservices.azure.com/ --body '
{
    "data_sources": [
      {
        "type": "azure_ml_index",
        "parameters": {
          "project_resource_id": "/subscriptions/<sub_id>/resourceGroups/<resource-group>/providers/Microsoft.MachineLearningServices/workspaces/<ml-workspace>",
          "name": "maroon-gyro-lw5ss1dmjt",
          "version": "1",
          "authentication": {
          "type": "access_token",
          "access_token": "'"$access_token"'"
        }
        }
      }
    ],
    "messages": [
      {
        "role": "user",
        "content": "What is the address in the current bill?"
      }
    ]
}
'

Make sure you give necessary permission to your service principal on the azure ml workspace.

Follow this for getting the token using a service principal.

Note: I would strongly recommend using a managed identity instead of a token.