Search code examples
kubernetesenvoyproxygrpc-web

How do I set up an envoy proxy in Minikube for a gRPC service?

I'm working on a little gRCP API and I've run into a small issue. I want the gRPC service to be accessible from a React front end, which means I need to have an envoy proxy that translates gRPC -> HTTP1 and vice versa.

TL;DR I think I can reach the envoy proxy, but the proxy isn't routing to my service correctly.

I'll put the info for the services here and explain what's happening below.

K8s (deployment and service) yaml files for each of my services:
Simulation deployment

Envoy config file and docker image
Envoy Config
Envoy Dockerfile

Here is a quick sanity check making sure that my service works, disregarding the envoy proxy:

> kubectl get services
NAME              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
kubernetes        ClusterIP   10.96.0.1        <none>        443/TCP          37d
sim-dep           NodePort    10.105.65.65     <none>        9090:30780/TCP   3s
sim-dep-http1     NodePort    10.111.190.170   <none>        8080:30948/TCP   3s
sim-envoy-proxy   NodePort    10.110.178.132   <none>        9091:32068/TCP   17h

> curl <minikube ip>:30948/v1/todo/all
{"api":"v1","toDos":[{}]}

> ./client-grpc -server=192.168.99.100:30780
<success from server, too much to put here but it works>

So from what I understand, this envoy proxy should accept connections on port 9091, then re-route them to the address sim-dep (K8s DNS is running) on port 9090.
Here is my code and error when I run this on a React app on my host machine (not in minikube). 

var todoService = new ToDoServiceClient(
   "http://192.168.99.100:32068/",
   null,
   null
);
var todo = new ToDo();
todo.setTitle("JS Created ToDo");
todo.setDescription("This todo was created with JS client");
var createRequest = new CreateRequest();
createRequest.setApi("v1");
createRequest.setTodo(todo);
var response = await todoService.create(createRequest, {}, <callback>);

Logs:

upstream connect error or disconnect/reset before headers

I'm assuming this means it can't connect at some level, but that error isn't very descriptive. I also can't curl through the envoy proxy, but I did try sending a request with PostMan and got this:

enter image description here

I'm not too sure what to make of that, but it sort of seems like it's reaching envoy but isn't reaching the service. I would appreciate any help on this! :) And let me know if I missed any info here. Thanks!

Solution

  • Just figured it out!

    To debug this I ran the proxy container locally. I was able to connect to it and was getting this error:

    unknown service /v1.ToDoService

    I hate myself for this, but that error was essentially saying this in english: 

    I can connect to your server, but I can't find a service called /v1.ToDoService

    and after a ridiculous amount of staring at my code and doing sanity checks, I realized that it was looking for the service in the wrong spot... the error really should give a bit more info but all I had to do was change the connection url in my JS client to this:

    http://localhost:32068

    No slash at the end! Whoops!

    Now to deal with deploying this proxy...

    So the deployment for the gRPC service has metadata on it that creates a DNS address(? not sure what its called) for the service. So in my case, I called my service sim-dep. If you follow the DNS section tutorial here, you can do a DNS lookup for your service which in my case gave me

    sim-dep.default.svc.cluster.local.

    Now all I had to do after finding that was replace the host address of the envoy config to that address and everything worked out!