Search code examples
node.jsnpmprotocol-buffersgrpcgrpc-node

gRPC Node Server: how to load protobuf file from a different Git repo?

Background:

  • we have multiple micro services maintained by different teams
  • most of the services are written in Java. but there is one written in Node.js
  • Node.js service interface is defined in .proto file and stored in the same repo as the Node.js code

Requirement

  • the Node.js server now needs to call another gRPC service (defined as service A) to get some values. In order to make the gRPC call, I need to first load the service definition.

Question:

  • Given that the service interface (.proto file) is defined in service A's repo, how should I load the proto file into my Node.js app? #Node.js #npm
  • what is the best way of managing inter-service communication? should my Node.js server call the other service just like how client calls a gRPC server?
Solution

  • A linux answer

    1) Given that the service interface (.proto file) is defined in service A's repo, how should I load the proto file into my Node.js app?

    In a development environment, a symbolic link to the proto directory in your protobuf folders will be enough.

    Let me give you an example. Let just say you have protobuf files in 2 repository and

    count.proto 

    syntax = "proto3";
package count.proto;
import "math/math.proto";
**Your proto code here**

    math.proto

    syntax = "proto3";
package math.proto
**Your proto code here this just contains messages no servies**

    They are in the respective directories.

    <path_to_repo_a>/proto/math/math.proto
<path_to_repo_b>/proto/count/count.proto

    Now you do a symbolic link to the root of your nodejs app(where package.json is).

    ln -s <path_to_repo_a>/proto repo_a_proto
ln -s <path_to_repo_b>/proto repo_b_proto

    Then you can now generate the proto files. In your root directory run

    mkdir -p generated_proto/math
mkdir -p generated_proto/count
// This command below is just to generate messages
protoc --proto_path=./repo_a_proto/ --js_out=import_style=commonjs,binary:generated_proto ./repo_a_proto/math/math.proto
// This generate both the messages and services
protoc --proto_path=./repo_b_proto/ -I./repo_a_proto/  --js_out=import_style=commonjs,binary:proto_ts --grpc_out=generated_proto/ --plugin=protoc-gen-grpc=`which grpc_node_plugin`  ./repo_b_proto/count/count.proto

    This is just tranlated to the path of the plugin binary . https://github.com/grpc/grpc-node

    which grpc_node_plugin

    In a production environment it will be just how you inject the folders with your ci/cd tools like teamcity or jenkins. Now you are done.

    what is the best way of managing inter-service communication? should my Node.js server call the other service just like how client calls a gRPC server?

    The answer is yes. It is just another microservice.