How to handle a json field, which can be one of two different types?

Question

I'm creating model classes for the MSCOCO annotation format.

There is one field in 'ObjectDetection', which can be one of two types:

annotation{
    "id": int, 
    "image_id": int, 
    "category_id": int, 
    "segmentation": RLE or [polygon], 
    "area": float, 
    "bbox": [x,y,width,height], 
    "iscrowd": 0 or 1,
}

So segmentation can either be a List<List<float>> or a RunLenghtEncoding, which as a class would look like this

public class RunLengthEncoding
{
    [JsonProperty("size")]
    public List<int>? Size { get; set; }

    [JsonProperty("counts")]
    public List<int>? Counts { get; set; }
}

Question is, how to handle this case when converting the json? Normally I would create an abstract class and inherit my two different types from that. Selection of the correct concrete type could be done in a custom converter.

However, with this configuration, this doesn't seem to be possible. I also don't want to use object.

Answer 1

In case future readers are interested, here is how I solved it at the moment:

In my Annotation model class I have these three properties for the segmentation:

[JsonProperty("segmentation")]
public JToken? Segmentation { get; set; }

[JsonIgnore]
public List<List<float>>? Polygons { get; set; }

[JsonIgnore]
public RunLengthEncoding? RLE { get; set; }

I then use the OnDeserialized callback to map the Segmentation to the correct property. In my case this is pretty easy, since according to the MSCOCO documentation RLE is used when IsCrowd is true, Polygons are used otherwise:

[OnDeserialized]
internal void OnDeserialized(StreamingContext context)
{
    if (Segmentation == null)
        return;

    if(IsCrowd)
    {
        RLE = Segmentation.ToObject<RunLengthEncoding>();
    }
    else
    {
        Polygons = Segmentation.ToObject<List<List<float>>>();
    }
}

Thanks again to @Heinzi for the suggestion!