How to export and import a fine-tuned AI model to a different instance of Azure AI Service or Azure OpenAI?

Question

I post here because Microsoft Q&A Azure when I post my question I get the 404 code error.

Let's imagine that I am working with different environments for Development, Test, and Production, each having its own resource group with AI resources, Azure OpenAI Service, or Azure AI Services.

Now, after fine-tuning a model by AI engineers, I want to deploy it to a target instance that is different from the source/training instance.

However, I cannot choose the target instance as the option is grayed out, here in Azure AI Studio. Here are my connected services:

In AzureOpenAI studio:

The target instance, whatever in the same or different region, is referenced by connection in Azure AI studio.

• Is it possible to deploy to a target instance that is different from the source instance?\

• Can it be automated by CI/CD pipelines?

• Where can I find documentation about this classic scenario?

By the way, for Azure Synapse Workspace, it is possible to generate an artifact/Synapse ARM JSON template and deploy it to various environments/instances.

Thanks for helping me out.

I tried multiple instance of hub and project for Azure AI Studio and also Azure OpenAI Studio but there is no option to deploy to other instance of service,

Answer 1

According to this documentation you deploy the fine-tuned model to different to a different subscription/region.

And below is the sample python code

import json
import os
import requests

token= os.getenv("<TOKEN>") 

subscription = "<DESTINATION_SUBSCRIPTION_ID>"  
resource_group = "<DESTINATION_RESOURCE_GROUP_NAME>"
resource_name = "<DESTINATION_AZURE_OPENAI_RESOURCE_NAME>"

source_subscription = "<SOURCE_SUBSCRIPTION_ID>"
source_resource_group = "<SOURCE_RESOURCE_GROUP>"
source_resource = "<SOURCE_RESOURCE>"


source = f'/subscriptions/{source_subscription}/resourceGroups/{source_resource_group}/providers/Microsoft.CognitiveServices/accounts/{source_resource}'

model_deployment_name ="gpt-35-turbo-ft" # custom deployment name that you will use to reference the model when making inference calls.

deploy_params = {'api-version': "2023-05-01"} 
deploy_headers = {'Authorization': 'Bearer {}'.format(token), 'Content-Type': 'application/json'}



deploy_data = {
    "sku": {"name": "standard", "capacity": 1}, 
    "properties": {
        "model": {
            "format": "OpenAI",
            "name": <"FINE_TUNED_MODEL_NAME">, # This value will look like gpt-35-turbo-0613.ft-0ab3f80e4f2242929258fff45b56a9ce 
            "version": "1",
            "source": source
        }
    }
}
deploy_data = json.dumps(deploy_data)

request_url = f'https://management.azure.com/subscriptions/{subscription}/resourceGroups/{resource_group}/providers/Microsoft.CognitiveServices/accounts/{resource_name}/deployments/{model_deployment_name}'

print('Creating a new deployment...')

r = requests.put(request_url, params=deploy_params, headers=deploy_headers, data=deploy_data)

print(r)
print(r.reason)
print(r.json())

and the same you can use it in you CI/CD pipelines.

For more information regarding fine-tuned model deployment go through this documentation.

You also mentioned

However, I cannot choose the target instance as the option is grayed out (in Azure AI Studio).

That is because the target region is not supported for deployment.

Below is the limitation for cross region/subscription deployment.

The only limitations are that the new region must also support fine-tuning and when deploying cross subscription the account generating the authorization token for the deployment must have access to both the source and destination subscriptions.