How to specify non-default Thrift Protocols and Transports

Question

I have read the official Apache Thrift docs. I have read the excellent Thrift By Example Docs. I have even read the even more excellent Thrift: The Missing Guide.

Nowhere can I figure out how to specify, say, TCompactProtocol over the default (TBinaryProtocol). Or, say, TFramedSocket over the default (TSocket).

Can someone elaborate here?

Answer 1

A protocol in Thrift describes the format how the actual data bits are written to/read from the underlying storage medium. Typically only one such thing is needed, stacking them is not necessary, since you only need one physical layout to serialize data.

In contrast, what is designed to be stacked are the so called "layered transports". Transports operate on a lower level of abstraction. Very typical examples are TFramedTransport and TBufferedTransport. Layered transports add functionality to the byte stream, such as adding a frame size, buffering data or offering multiplexing capabilities. Technically these are located between protocol and the underlying "endpoint transports" which actually write/read the bytes to/from a storage medium (e.g. a Socket).

Consequently, if you need to change the conversion from and into bytes, you write another protocol. Whenever an additional storage medium is required, write an endpoint transport. If the bytes should only be manipulated in between of the serialization/deserialization process, say you want to add your new compression algorithm "Schrippe" that lets Brotli and Zopfli look like old bread (pun very intended), write a layered transport.

A good example how to stack everything together can be found in the language-dependent implementation of the Thrift Test Suite (with a few exceptions located under /test/yourlanguage). The test suite is designed to test all kinds of combinations. If you are done with the tutorial, I recommend to take a look into that code, as it explains a lot of the advanced features for both the client end and the server end.

Just as an example, this is the relevant part from the C++ Test Client where all selected components are set up:

  boost::shared_ptr<TTransport> transport;
  boost::shared_ptr<TProtocol> protocol;

  boost::shared_ptr<TSocket> socket;
  boost::shared_ptr<TSSLSocketFactory> factory;

  if (ssl) {
    factory = boost::shared_ptr<TSSLSocketFactory>(new TSSLSocketFactory());
    factory->ciphers("ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH");
    factory->loadTrustedCertificates((dir_path + "../keys/CA.pem").c_str());
    factory->authenticate(true);
    socket = factory->createSocket(host, port);
  } else {
    if (domain_socket != "") {
      if (abstract_namespace) {
        std::string abstract_socket("\0", 1);
        abstract_socket += domain_socket;
        socket = boost::shared_ptr<TSocket>(new TSocket(abstract_socket));
      } else {
        socket = boost::shared_ptr<TSocket>(new TSocket(domain_socket));
      }
      port = 0;
    } else {
      socket = boost::shared_ptr<TSocket>(new TSocket(host, port));
    }
  }

  if (transport_type.compare("http") == 0) {
    boost::shared_ptr<TTransport> httpSocket(new THttpClient(socket, host, "/service"));
    transport = httpSocket;
  } else if (transport_type.compare("framed") == 0) {
    boost::shared_ptr<TFramedTransport> framedSocket(new TFramedTransport(socket));
    transport = framedSocket;
  } else {
    boost::shared_ptr<TBufferedTransport> bufferedSocket(new TBufferedTransport(socket));
    transport = bufferedSocket;
  }

  if (protocol_type.compare("json") == 0) {
    boost::shared_ptr<TProtocol> jsonProtocol(new TJSONProtocol(transport));
    protocol = jsonProtocol;
  } else if (protocol_type.compare("compact") == 0) {
    boost::shared_ptr<TProtocol> compactProtocol(new TCompactProtocol(transport));
    protocol = compactProtocol;
  } else if (protocol_type == "header") {
    boost::shared_ptr<TProtocol> headerProtocol(new THeaderProtocol(transport));
    protocol = headerProtocol;
  } else {
    boost::shared_ptr<TBinaryProtocol> binaryProtocol(new TBinaryProtocol(transport));
    protocol = binaryProtocol;
  }

  // Connection info
  cout << "Connecting (" << transport_type << "/" << protocol_type << ") to: ";
  if (abstract_namespace) {
    cout << '@';
  }
  cout << domain_socket;
  if (port != 0) {
    cout << host << ":" << port;
  }
  cout << endl;