Search code examples
c#jsonserializationdeserializationjsonlines

C# Type safe JSON-Lines Deserialization


Currently I am working with the Shopify GraphQL Bulk Query.
This Query returns a JSON Lines file. Such a file may look like this:

{"id":"gid:\/\/shopify\/Product\/5860091625632","title":"Levis Jeans","description":"Cool Jeans","vendor":"Levis","status":"ACTIVE"}
{"id":"gid:\/\/shopify\/ProductImage\/20289865679008","__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178118963360","title":"32","position":1,"image":null,"selectedOptions":[{"name":"Size","value":"32"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":10,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118963360"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178118996128","title":"31","position":2,"image":null,"selectedOptions":[{"name":"Size","value":"31"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":5,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118996128"}
{"available":3,"location":{"id":"gid:\/\/shopify\/Location\/57951518880"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118996128"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178119028896","title":"34","position":3,"image":null,"selectedOptions":[{"name":"Size","value":"34"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":5,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178119028896"}
{"available":15,"location":{"id":"gid:\/\/shopify\/Location\/57951518880"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178119028896"}

Each line of this file is a valid JSON-object and the lines are connected via __parentId with each other.
My Goal is to Deserialize this into C# Classes like this:

class Product
{
    public string Id { get; set; }
    public string Title { get; set; }
    public string Description { get; set; }
    public IEnumerable<ProductImage> Images { get; set; }
    public IEnumerable<ProductVariant> Variants { get; set; }
}
class ProductImage
{
    public string Id { get; set; }
}
class ProductVariant
{
    public string Id { get; set; }
    public IEnumerable<IDictionary<string, string>> SelectedOptions { get; set; }
    public IEnumerable<InventoryLevel> Levels { get; set; }
}
class InventoryLevel
{
    public int Available { get; set; }
} 

And the output of a potential function performing the deserialization:

var file = new System.IO.StreamReader(@"c:\test.jsonl");
var products = DeserializeJsonL<IEnumerable<Product>>(file);

Shopify suggests to read the file in reverse. I get the Idea. But I cannot imagine how to deserialize this file in a type safe way. How could I determine if the current line is a ProductVariant, a ProductImage or something else? I cannot influence the JSONL Output to include type information.

I am pretty sure without type information I cannot deserialize it safely. But how should I handle this data then to insert into a database for example?

EDIT the classname in {"id":"gid:\/\/shopify\/Product\/5860091625632"} cannot be used to determine the Type!


Solution

  • I ended up adding some sort of type information to my graphql-query by defining a unique fieldname for each type which may be on a new line in the resulting JSON Lines file.

    For that i used GraphQL field aliases:

    someQuery {
       uniqueFieldAlias : fieldName
    }
    

    When i read the file i search on each line for the unique fieldname. Then i deserialize the line into the corresponding class.

    using (var file = new StreamReader(await res.Content.ReadAsStreamAsync()))
    {
        string line;
    
        while ((line = await file.ReadLineAsync()) != null)
        {
            if (line.Contains("\"uniqueFieldAlias\""))
            {
                var product = JsonSerializer.Deserialize<Product>(line);
                products.Add(product);
                continue;
            }
            if (line.Contains("\"otherUniqueAlias\""))
            {
                var somethingElse = JsonSerializer.Deserialize<SomeClass>(line);
                products[productIndex].Something.Add(somethingElse);
                continue;
            }
        }
    }
    

    The idea is inspired by @Caius Jard comments