Search code examples
rpcgrpcgrpc-java

grpc complete async java service request/response mapping

A Java service (let's call it portal) is both a gRPC client as well as server. It serves millions of gRPC clients (server), each client requesting for some task/resource. Based on the incoming request, portal will figure out the backend services and talk to one or more of them, and send the returned response(s) to the originating client. Hence, here, the requirement is:

  1. Original millions of clients will have their own timeouts
  2. The portal should not have a thread blocking for the millions of clients (async). It should also not have a thread blocking for each client's call to the backend services (async). We can use the same thread which received a client call for invoking the backend services.
  3. If the original client times out, portal should be able to communicate it to the backend services or terminate the specific call to the backend services.
  4. On error from backend services, portal should be able to communicate it back to the specific client whose call failed.

So the questions here are:

  1. We have to use async unary calls here, correct?
  2. How do the intermediate server (portal) match the original requests to the responses from the backend services?
  3. In case of errors on backend services, how does the intermediate server propagate the error?
  4. How does the intermediate server propagate the deadlines?
  5. How does the intermediate server cancel the requests on the backend services, if the originating client terminates?
Solution

  • gRPC Java can make a proxy relatively easily. Using async stubs for such a proxy would be common. When the proxy creates its outgoing RPCs, it can save a reference to the original RPC in the callback of the outgoing RPC. When the outgoing RPC's callback fires, simply issue the same call to the original RPC. That solves both messages and errors.

    Deadline and cancellation propagation are automatically handled by io.grpc.Context.

    You may want to reference this grpc-level proxy example (which has not been merged to grpc/grpc-java). It uses ClientCall/ServerCall because it was convenient and because it did not want to parse the messages. It is possible to do the same thing using the StreamObserver API.

    The main difficulty in such a proxy would be to observe flow control. The example I referenced does this. If using StreamObserver API you should cast the StreamObserver passed to the server to ServerCallStreamObserver and get a ClientCallStreamObserver by passing a ClientResponseObserver to the client stub.