Search code examples
javaspring-bootkubernetesmicroserviceskubernetes-ingress

Best way for inter-cluster communication between microservices on Kubernetes?

I want to understand what is the best way I can implement below behaviour in microservices deployed on Kubernetes :

There are 2 different K8s clusters. Microservice B is deployed on both the clusters.

Now if a Microservice A calls Microservice B and B’s pods are not available in cluster 1, then the call should go to B of cluster 2.

I could have imagined to implement this functionality by using Netflix OSS but here I am not using it.

Also, keeping aside the inter-cluster communication aside for a second, how should I communicate between microservices ?

One way that I know is to create Kubernetes Service of type NodePort for each microservice and use the IP and the nodePort in the calling microservice.

Question : What if someone deletes the target microservice's K8s Service? A new nodePort will be randomly assigned by K8s while recreating the K8s service and then again I will have to go back to my calling microservice and change the nodePort of the target microservice. How can I decouple from the nodePort?

I researched about kubedns but seems like it only works within a cluster.

I have very limited knowledge about Istio and Kubernetes Ingress. Does any one of these provide something what I am looking for ?

Solution

  • You can expose you application using services, there are some kind of services you can use:

    • ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType.

    • NodePort: Exposes the Service on each Node’s IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You’ll be able to contact the NodePort Service, from outside the cluster, by requesting <NodeIP>:<NodePort>.

    • LoadBalancer: Exposes the Service externally using a cloud provider’s load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created.

    • ExternalName: Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record

    For internal communication you an use service type ClusterIP, and you could configure the service dns for your applications instead an IP. I.e.: a service called my-app-1 could be reach internnaly using the dns http://my-app-1 or with fqdn http://my-app-1.<namespace>.svc.cluster.local.

    For external communication, you can use NodePort or LoadBalancer.

    NodePort is good when you have few nodes and know the ip of all of them. And yes, by the service docs you can specify a specific port number:

    If you want a specific port number, you can specify a value in the nodePort field. The control plane will either allocate you that port or report that the API transaction failed. This means that you need to take care of possible port collisions yourself. You also have to use a valid port number, one that’s inside the range configured for NodePort use.

    LoadBalancer give you more flexibility, because you don't need to know all node ips, you just must to know the service IP and port. But LoadBalancer is only supported in cloudproviders, if you wan to implement in bare-metal cluster, I recomend you take a look in MetalLB.

    Finally, there is another option that is use ingress, in my point of view is the best way to expose HTTP applications externally, because you can create rules by path and host, and it gives you much more flexibility than services. But only HTTP/HTTPS is supported, if you need TCP then go to Services.

    I'd recommend you take a look in these links to understand in deep how services and ingress works:

    Kubernetes Services

    Kubernetes Ingress

    NGINX Ingress