There are some great libraries out there for deserializing binary formats. I really like the declarative approach by kaitai and nom's approach which is using Rust.
However, I am not aware of any good approaches to serialize binary formats.
For example, you often have the case that you have to write your message length right into the message header, but actually you do not know your exact message length at this point because it depends on many fields which are downstream from the header. And you sometimes also have to deal with padding alignment which can be cumbersome.
Do you know any solutions for problems like these?
Please take a look at ASN.1 which has solved this problem many years ago, and is still continuing to be widely used in critical infrastructure in many different industries. It is independent of programming language and machine architecture so you can set up communication whether one peer is using C on a little-endian machine and the other is using Java or C# on a big-endian machine. Structure padding issues are easily handed by good quality tools for ASN.1. A good list of tools (both free and commercial) is available at the ASN.1 Tools page of the ITU-T ASN.1 Project.