Azure Search v11: Indexing nullable Collection of Complex Type

Question

I'm updating the SDK for the Azure Cognitive Search service, from v10 to v11. I have followed all the steps in the guide in order to upgrade, however I have noticed a strange behavior about the indexing (merge or upload) operation: the UploadDocumentAsync (but also with other methods used to indexing data) operation fails when a property of type Collection (Edm.ComplexType) is null, with the following error:

A node of type 'PrimitiveValue' was read from the JSON reader when trying to read the contents of the property. However, a 'StartArray' node was expected json.

IndexDocumentsResult response = await searchClient.UploadDocumentsAsync<T>(documents).ConfigureAwait (false);

With v10 this problem did not arise. A workaround I found is to set collections as empty arrays and not with null value, but I would like to find a better solution.

EDITED: I upgraded from Microsoft.Azure.Search v10.1.0 to Azure.Search.Documents v11.1.1 Following an example of a generic T class used to indexing data:

public class IndexEntity
{
    [JsonProperty("@search.score")]
    public double SearchScore { get; set; }

    [JsonProperty("Key")]
    public Guid Id { get; set; }

    [JsonProperty("Code")]
    public string Code { get; set; }

    [JsonProperty("ComplexObj")]
    public ComplexType[] CollectionOfComplexType{ get; set; }

}

Following the definition of ModelObjectToIndex

public class ComplexType
{
    [JsonProperty("Id")]
    public string Id { get; set; }

    [JsonProperty("Value")]
    public string Value { get; set; }
}

Basically when the CollectionOfComplexType property is null, I get the above error. If I set it as an empty array, the error does not occur, but as mentioned I don't like this solution, furthermore in the old version it was an allowed operation (the indexing was completed successfully)

Answer 1

Our Azure.Search.Documents behavior seems to have changed in this regard. I've opened https://github.com/Azure/azure-sdk-for-net/issues/18169 to track resolution.

You can workaround this issue without initializing your collections to an empty array by passing in a JsonSerializerSettings that was similar to what we did in our older Microsoft.Azure.Search library, since it seems from using the JsonPropertyAttribute you're using Newtonsoft.Json (aka Json.NET) anyway:

1. Add a package reference to Microsoft.Azure.Core.NewtonsoftJson if you haven't already. It recently GA'd so you don't need to use a preview if you were, which I presume since System.Text.Json - our default serializer - would not have honored your property renames.

2. Pass in a JsonSerializerSettings before creating your SearchClient like so:

   var settings = new JsonSerializerSettings
   {
       // Customize anything else you want here; otherwise, defaults are used.
       NullValueHandling = NullValueHandling.Ignore,
   };
   
   var options = new SearchClientOptions
   {
       Serializer = new NewtonsoftJsonObjectSerializer(settings),
   };
   
   var searchClient = new SearchClient(options);
   

We'll discuss how to resolve this by default, if we even can. One big change from the older library is the ability to customize the serializer used. By default we use System.Text.Json, but we support other serializers including Newtonsoft.Json. If someone were to pass in their own settings - or even desire the defaults - changing that could be catastrophic. So I'm curious: if we at least documented this behavior change (perhaps on the SearchClient class remarks and/or UploadDocuments and related methods) and how to retain previous behavior, would that have helped or otherwise been satisfactory?