Deserialize from Json using existing sub objects

Question

Here is my situation: I am trying to use Newtonsoft.Json 13.0.3 to deserialize by using already instanced data.

I know this exists: JsonConvert.PopulateObject but that does not do the trick, bare with me.

// MainData, a class we will let Json.net instanciate
public class MainData
{
  [JsonProperty]
  private List<SmallerData> _smallerDataArray = new();
}

// SmallerData, a class we do not want Json.net to instanciate
public class SmallerData
{
   public int myInt;
}

Imagine the following MainData serialized into a json string called "jsonString"

{
  "$type":"MainData",
  "_smallerDataArray":
  {
    "$type":"List<SmallerData>"
    [
      {
        "$type":"SmallerData",
        "myInt":"0"
      },
      {
        "$type":"SmallerData",
        "myInt":"1"
      },
      {
        "$type":"SmallerData",
        "myInt":"2"
      }
    ]
  }
}

Normally deserialisation code looks like that

public static MainData DeserializeNormaly(string jsonString)
{    
   // here Json.net will create 3 SmallerData instances via RTTI
   return JsonConvert.DeserializeObject<MainData>(jsonString);
}

But I would like it to look like something like that (that code does not compile)

public class Example
{
   // Creating a pool of 3 SmallerData for the example's sake
   List<SmallerData> _pooledData = new() { new SmallerData(), new SmallerData(), new SmallerData());

  public object MyObjectFactory(Type objectType)
  {
     if(objectType == typeof(SmallerData))
     {
       // Detecting a type we can instaniate from a pool
       SmallerData smallerDataInstance = _pooledData[0];
       _pooledData.RemoveAt(0);
       return smallerDataInstance;
     }
     // Relying of default object creation
     return null;
  }

  public MainData DeserializeFromPool(string jsonString)
  {   
  
   JsonSerializerSettings settings = new();
   settings.ObjectFactory = MyObjectFactory;

   // here Json.net will create 3 SmallerData instances via RTTI
   return JsonConvert.DeserializeObject<MainData>(jsonString, settings);
  }
}

Somehow I would like JsonConvert.DeserializeObject to pull from pooledData when instanciating SmallerData instead of creating new ones.

I have been looking into converters contracts what whatnots but couldn't find a way. Anybody attempted that before ?

In the case you are curious as to why I am trying to do that: this one of the many efforts I am doing to reduce garbage generation during json usage.

Answer 1

Update If you need to support TypeNameHandling during deserialization you cannot use a CustomCreationConverter<T> as converters are required to handle everything themselves including metadata type resolution.

Instead, create a custom contract resolver, override DefaultContractResolver.CreateObjectContract() and set DefaultCreator to pull objects from your pool as shown in this answer to Type resolution in Newtonsoft.Json with custom converter.

Original answer You can use a CustomCreationConverter<SmallerData> to pull your objects from their memory pool.

Let's say your SmallerData looks something like this, with a shared pool across all threads:

public class SmallerData
{
    public static ObjectPool<SmallerData> Pool { get; } = new();

    public static SmallerData Create() => Pool.TryRemove(out var o) ? o : new();
}

public class ObjectPool<TObject>
{
    readonly ConcurrentQueue<TObject> queue = new(); // I chose a queue to get FIFO behavior but if you want LIFO use ConcurrentStack<T>
    
    public void Add(TObject obj) => queue.Enqueue(obj ?? throw new ArgumentNullException(nameof(obj)));
    public void AddRange(params TObject [] objs) => Array.ForEach(objs, o => Add(o));
    public bool TryRemove( [System.Diagnostics.CodeAnalysis.NotNullWhen(returnValue: true)] out TObject? obj) => queue.TryDequeue(out obj) && obj is not null;
}

Then you can define a converter as follows:

public class SmallerDataConverter : CustomCreationConverter<SmallerData>
{
    public override SmallerData Create(Type objectType) => SmallerData.Create();
}

And deserialize by adding the converter to settings:

SmallerData [] itemsToPool = [new(), new(), new()];
SmallerData.Pool.AddRange(itemsToPool);

var settings = new JsonSerializerSettings 
{
    Converters = { new SmallerDataConverter() },
};

var root = JsonConvert.DeserializeObject<MainData>(jsonString, settings);

Or apply the converter directly to SmallerData using attributes to ensure your data class is constructed using the pool whenever it is deserialized:

[JsonConverter(typeof(SmallerDataConverter))]
public class SmallerData
{
    public static ObjectPool<SmallerData> Pool { get; } = new();

    public static SmallerData Create() => Pool.TryRemove(out var o) ? o : new();
}

Either way, pooled instances will be used during deserialization. Demo fiddle here: https://dotnetfiddle.net/CX0OTc

Alternatively, you could eliminate the global pool and make a pool local to each converter, like so:

public class SmallerDataConverter : CustomCreationConverter<SmallerData>
{
    public ConcurrentQueue<SmallerData> Queue { get; } = new(); // I chose a queue to get FIFO behavior but if you want LIFO use ConcurrentStack<T>

    public SmallerDataConverter(params SmallerData [] objs) => Array.ForEach(objs, o => Queue.Enqueue(o ?? throw new ArgumentNullException(nameof(o))));

    public override SmallerData Create(Type objectType) => Queue.TryDequeue(out var o) && o is not null ? o : new();
}

However, with this approach you will be unable to apply the converter via attributes and will need to pass the items to be pooled into the converter constructor, like so:

SmallerData [] itemsToPool = [new(), new(), new()];
var settings = new JsonSerializerSettings 
{
    Converters = { new SmallerDataConverter(itemsToPool) },
};

var root = JsonConvert.DeserializeObject<MainData>(jsonString, settings);

Even if the pool is local to a given converter it is a good idea to make it thread-safe as many applications share settings across threads.

Demo fiddle #2 here.