Search code examples
microservicesgrpchot-reload

One giant servicer of many services vs. one servicer per service

Setup

  • My app uses gRPC for frontend/backend separation
  • Backend (server + services) is written in Python

Goals

I'd love to achieve live-updates including:

  1. Updating existing services
  2. Adding new services
  3. Removing services

Problem

I can achieve #1 by using Python's importlib feature. But there are fewer options for adding/removing services. It seems to depend on the app's gRPC implementation. The major constraints seem to be the fact that servicers can only be registered before running the server, i.e., through the call to

add_MyServiceServicer_to_server()

So does adding reflection support, which is through the call to

service_names = [
    MyService_pb.DESCRIPTOR.services_by_name[''].full_name,
    ...
]

reflection.enable_server_reflection(service_names, my_server)

Solution Candidates

  • Approach 1: Have one servicer per service, much like the official SayHello example of gRPC
  • Approach 2: Have one giant servicer that includes all the other services as its RPC methods.

Approach 1 seems to be intuitive, but it won't support adding/removing services while the server is running.

Approach 2 seems promising, but it is confusing by sticking the entire universe in a single servicer. And I'm not sure how gRPC's thread pool would like this approach.

Questions

  • Can I achieve my goals with these approaches?
  • If true, which one is better in terms of both maintainability and performance?
  • If false, are there alternatives?
Solution

  • As per @DougFawley's comments,

    Typically microservices would have multiple replicas deployed, and be restarted in a rolling update when new services are added.

    "What's the best user experience like in this scenario?" -> microservice clients should expect and be resilient to RPC failures. They can happen for many other reasons in steady state. Typically you will run multiple replicas and when one is restarted, if you gracefully shut it down, clients will have a chance to create connections to other backends and no RPCs will fail. But if they do fail, clients should retry and will use another backend. Users should not really be impacted by this.

    In short, it's a bad idea to add/remove services without rebooting server. So for this reason, I'd better adopt the one-to-one servicer-service binding and not hack it for this particular live-update intent.