Architectural Decision: How to structure a big Json Response

Question

I'm working on an app that will generate a Json potentially very big. In my tests this was 8000 rows. This is because is an aggregation of data for a year, and is required to display details in the UI.

For example:

"voice1": {
    "sum": 24000,
    "items": [
        {
            "price": 2000,
            "description": "desc1",
            "date": "2021-11-01T00:00:00.000Z",
            "info": {
                "Id": "85fda619bbdc40369502ec3f792ae644",
                "address": "add2",
                "images": {
                    "icon": "img.png",
                    "banner": null
                }
            }
        },
        {
            "price": 2000,
            "description": "desc1",
            "date": "2021-11-01T00:00:00.000Z",
            "info": {
                "Id": "85fda619bbdc40369502ec3f792ae644",
                "address": "add2",
                "images": {
                    "icon": "img.png",
                    "banner": null
                }
            }
        }
    ]
},

The point is that I can have potentially 10 voices and for each dozen and dozens of items.

I was wondering if you can point to me some Best Practices or if you have some tips about them because I've got the feeling this can be done better.

Answer 1

It sounds like you are finding out that JSON is a rather verbose format (not as bad as XML but still very verbose). If you are worried about the size of messages between server client and you have a few options:

1. JSON compresses rather well. You can see how most tokens repeat many times. So make sure to Gzip or Snappy before sending to clients. This will drastically reduce the size, but cost some performance for inflating / deflating.

2. The other alternative is to not use JSON for transfer, but a more optimized format. One of the best options here is Flat Buffers. It does require you to provide schemas of the data that you are sending but is an optimized binary format with minimal overhead. It will also drastically speed up your application because it will remove the need for serialization / deserialization, which takes a significant time for JSON. Another popular, but slightly slower alternative is Protobuf.