Search code examples
optimizationserializationdeserializationbyteprotocol-buffers

Do bytebuffer serializers optimized to the bit level (simillar to protobuff) exist?


I work at an environment where alot of data is passed via udp, serialized to bytes. Currently we work hard to optimize our structures to be efficient, but it comes with a cost - time and serialization/desrialization bugs.

I searched the web for something simillar to Google's Protobuff in an attempt to make the process more seamless and less error prone.

it doesn't have to be 100% optimized, but we still need to optimize more than protobuff.

Does something like this exist? Maybe where it's possible in the schema to specify either range or the number of bits?

thank you

edit: Ideally it would support Java, C, CPP and Python. But even if it supports only one of them it will still be helpful.


Solution

  • Have you considered ASN.1 serialization? It supports everything you are asking for here. You can place value range constraints on components, or place size constraints on various types. It is independent of programming language or machine architecture. There are tools available for C, C++, C#, Java and Python, so you can choose which ever language you prefer to work with.

    A great list of ASN.1 Tools can be found at https://www.itu.int/en/ITU-T/asn1/Pages/Tools.aspx including both free and commercial tools.