Azure search schema migrations

Question

What's the best way to migrate an Azure search schema in a release pipeline?

In the SQL world, I'd use something like DbUp. Is there anything similar for Azure search? Or is there a different approach when the schema needs to change?

Answer 1

It depends on whether you are pushing content via the SDK or if you are pulling content from a supported content source using one of the pre-built indexers. Using the SDK you can add new properties to your model as explained in this post: Update Azure search document schema

Note: Changes in your data model may require either an update or a rebuild of the index. For example, adding new properties only requires and update. But, if you change a setting like searchable, filterable or sortable you will need a rebuild. For details, see How to rebuild an index in Azure Cognitive Search

COMPATIBILITY PROBING VIA PUSH

My preferred solution is to use push indexing for everything. To test if the data model in the index is compatible, I create a sample item and submit to the index. If the model used in the index is incompatible, and error is thrown by the Azure Search SDK. I then delete the index and create it from scratch using my new model.

Here is a simplified method to test if any item is compatible with a named index:

public async Task<bool> TestItemAsync<T>(T item, string indexName)
{
    var isCompatible = false;
    var indexClient = _searchServiceClient.Indexes.GetClient(indexName);
    var indexActions = new List<IndexAction<T>> { IndexAction.MergeOrUpload((item)) };

    var batch = IndexBatch.New(indexActions);
    try
    {
        var unused = await indexClient.Documents.IndexAsync(batch);
        isCompatible = true;
    }
    catch
    {
        // ignored
    }

    return isCompatible;
}

And here is an example of how you might use it to probe your index for compatibility.

var item = new Product();
item.ID = 123;
item.Title = "Sample";
item.MyNewProperty = "Some value"
// ... 

var isCompatible = await TestItemAsync(item, indexName);
if (isCompatible)
{
    await DeleteItemAsync(item, indexName);
}
else 
{
    await DeleteIndexAsync<T>(indexName);
    await CreateIndexAsync<T>(indexName);
}

To actually verify the compatiblity, make sure you populate your item with some values. In this example I hard-coded some values for the sake of the example. In actual use I use a Mock framework that populates every property of my item with some random sample value, but I did not want to give the impression that a third-party component is required for this use-case.