Search code examples
c.net-corethrift

How to use Thrift for message passing instead of RPC


I know that Thrift is mainly aimed at full fledged client-server RPC but looking at the high level architecture it seems to me that it should be perfectly usable for bidirectional message passing as well.

What I would like to build on both ends (C, .NET Core) is following:

  • Receive method: has reference to a socket, reads a full message, deserializes it, returns it
  • Send method: has reference to a socket, serializes a given message, sends it to the wire.

I don't need threading servers, ... anything fancy. Essentially what I would love to get on top of what e.g. Protobuffs offers is out-of-box handling of buffering the whole message on receive end & generally message framing.

The problem is that I wasn't able to find any documentation on how to start building that using the current libs (I'm personally interested in .NET Core and C ones) APIs. The only thing I've found is this question but it doesn't really point to any resources.


Solution

  • Thrift is an RPC- and serialization framework. That implies, that you also may use only the serialization part without RPC.

    In combination with messaging systems the way to go is usually (roughly) as follows:

    • serialize the message into a buffer
    • send that buffer by whatever means you want
    • the receiving end(s) deserialize(s) the buffer and process the data

    If you plan to send different kinds of messages through the same channel, it may be a good idea to have a union envelope structure that contains all possible message bodies:

     struct MessageOne {
          // contents of this message type
     }
    
     struct MessageTwo {
          // contents of this message type
     }
    
     struct MessageThree {
          // contents of this message type
     }
    
     union MyMessageEnvelope {
          1: MessageOne   one
          2: MessageTwo   two
          3: MessageThree  three
          // easily extendable 
     }
    

    To make it more elegant/reusable, one could also implement a custom transport to fit the needs and to encapsulate the logic a bit more. Thrift's modular structure makes it easy (the post linked by you also refers to that). There are a few samples in the /contrib folder of the source tree which may serve as a starting point.

    If you have absolutely no idea where to start: Look at the tutorial and then at the test suite programs, both are great as a learning resource for Thrift starters.