Search code examples
c#jsonserializationjson.netsystem.memory

How to deserialize system.memory with JsonConvert.DeserializeObject in c#


I have a class which uses a System.Memory<double> Property, let's call it PROP1 and CLASS1

When this class is serialized into a JSON file it's saved in a pretty common way :

(...) "PROP1":[7200.0,7200.0,7200.0] (...)

When I try to deserialize via JsonConvert.DeserializeObject<CLASS1>File.ReadAllText(fileName));

I get the following exception :

Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'System.Memory`1[System.Double]' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly. To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.

I guess it cannot serialize it because Memory is working more like a pointer, so you should create an object first which the Memory refers to. But I could not find a fix to this (besides rewriting the class to another type)...and I couldn't find any threads with a similar problem. Any ideas how to deserialize it??


Solution

  • Since Json.NET doesn't seem to have built-in support for serializing and deserializing Memory<T> and ReadOnlyMemory<T>, you could create generic converters for them that will serialize and deserialize "snapshots" of the contents of Memory<T> and ReadOnlyMemory<T> slices as follows:

    public class MemoryConverter<T> : JsonConverter<Memory<T>>
    {
        public override void WriteJson(JsonWriter writer, Memory<T> value, JsonSerializer serializer) =>
            serializer.SerializeMemory((ReadOnlyMemory<T>)value, writer, false);
            
        public override Memory<T> ReadJson(JsonReader reader, Type objectType, Memory<T> existingValue, bool hasExistingValue, JsonSerializer serializer) =>
            reader.MoveToContentAndAssert().TokenType switch
            {
                JsonToken.String when typeof(T) == typeof(char) => 
                    ((T [])(object)serializer.Deserialize<string>(reader).ToCharArray()).AsMemory(),
                JsonToken.StartArray when typeof(T) == typeof(byte) => 
                    ((T [])(object)serializer.Deserialize<List<byte>>(reader).ToArray()).AsMemory(),
                _ => 
                    serializer.Deserialize<T []>(reader).AsMemory()
            };
    }
    
    public class ReadOnlyMemoryConverter<T> : JsonConverter<ReadOnlyMemory<T>>
    {
        public override void WriteJson(JsonWriter writer, ReadOnlyMemory<T> value, JsonSerializer serializer) =>
            serializer.SerializeMemory((ReadOnlyMemory<T>)value, writer, serializeAsString : true);
            
        public override ReadOnlyMemory<T> ReadJson(JsonReader reader, Type objectType, ReadOnlyMemory<T> existingValue, bool hasExistingValue, JsonSerializer serializer) =>
            reader.MoveToContentAndAssert().TokenType switch
            {
                JsonToken.String when typeof(T) == typeof(char) => 
                    (ReadOnlyMemory<T>)(object)serializer.Deserialize<string>(reader).AsMemory(),
                JsonToken.StartArray when typeof(T) == typeof(byte) => 
                    ((T [])(object)serializer.Deserialize<List<byte>>(reader).ToArray()).AsMemory(),
                _ => 
                    serializer.Deserialize<T []>(reader).AsMemory()
            };
    }
    
    public static partial class JsonExtensions
    {
        internal static void SerializeMemory<T>(this JsonSerializer serializer, ReadOnlyMemory<T> value, JsonWriter writer, bool serializeAsString) 
        {
            switch (value)
            {
                case ReadOnlyMemory<byte> m when MemoryMarshal.TryGetArray(m, out var seg) && seg.Offset == 0 && seg.Count == seg.Array.Length:
                    writer.WriteValue(seg.Array); // Base64 encoded array.
                break;
                case ReadOnlyMemory<byte> m:
                    writer.WriteValue(m.ToArray()); // Base64 encoded slice.
                break;
                case ReadOnlyMemory<char> m when serializeAsString:
                    writer.WriteValue(m.ToString());
                break;
                default:
                    serializer.Serialize(writer, MemoryMarshal.ToEnumerable(value));
                break;
            }
        }
    
        public static JsonReader MoveToContentAndAssert(this JsonReader reader)
        {
            ArgumentNullException.ThrowIfNull(reader);
            if (reader.TokenType == JsonToken.None)       // Skip past beginning of stream.
                reader.ReadAndAssert();
            while (reader.TokenType == JsonToken.Comment) // Skip past comments.
                reader.ReadAndAssert();
            return reader;
        }
    
        public static JsonReader ReadAndAssert(this JsonReader reader)
        {
            ArgumentNullException.ThrowIfNull(reader);
            if (!reader.Read())
                throw new JsonReaderException("Unexpected end of JSON stream.");
            return reader;
        }
    }
    

    Then you would serialize and deserialize your model using the following settings:

    var settings = new JsonSerializerSettings
    {
        Converters = { new MemoryConverter<double>(), new ReadOnlyMemoryConverter<double>() },
    };
    
    var json = JsonConvert.SerializeObject(class1, settings);
    var model2 = JsonConvert.DeserializeObject<CLASS1>(json, settings);
    

    And/or apply to your model as follows:

    public class CLASS1
    {
        [JsonConverter(typeof(MemoryConverter<double>))]
        public Memory<double> PROP1 { get; set; }
    };
    

    Warnings and notes:

    • Warning: references of array slices are not preserved. If your Memory<double> is a slice of some array, and that array is also being serialized elsewhere in the model, then when deserialized the Memory<double> will not refer to the deserialized array by reference, it will refer to its own copy of the array.

      If you need to preserve the references of array slices, a different (and much more complicated) converter would be required.

    • Since byte arrays are serialized as Base64 strings by Json.NET (and System.Text.Json), I did the same for Memory<byte> and ReadOnlyMemory<byte>. But if the Memory<byte> had been previously serialized as a JSON array, it will be read properly.

    • Since ReadOnlyMemory<char> can sometimes wrap a string, I serialized it as such, but did not do the same for Memory<char> which can only wrap a char array.

      If you don't want that, pass serializeAsString : false inside ReadOnlyMemoryConverter<T>.Read().

    • Absent the converters, I was unable to generate a reasonable serialization for Memory<T> out of the box with Json.NET 13.0.3. Instead of a JSON array, I got

      {"PROP1":{"Length":6,"IsEmpty":false}}
      

      Maybe you already have some converter installed that handles serialization but not deserialization?

    Demo fiddle here.