Search code examples
node.jskubernetesmicroserviceskubernetes-ingresskubectl

How does Kubernetes distribute "services" to each node?


I'm new to Kubernetes world. So far what I understand is that Kubernetes utilizes multiple nodes (server machines) to implement micro-service. The thing I don't get is that how do I make Kubernetes assign certain Service object (with associated Deployment object) to certain nodes?

For better clarification of my question, let's say I have 100 different worker nodes (server machines) with different specs for each (CPU, RAM, etc). And let's say I have a website application (aka. webapp) I want to host as microservice using those 100 nodes via Kubernetes. This webapp has multiple "components" such as frontend, backend, socketio server, database, etc.

Ok, so now I create Deployment and Service objects for each component and apply using kubectl apply -f .. command. And here is where it gets confusing for me.

So how does Kubernetes assign each webapp component to certain node? Is this done automatically? If it's done automatically, how do I know how many nodes will be used and which component will be deployed to which nodes amongst 100 nodes I have (e.g. 30 nodes with frontend, 20 nodes with backend, etc)? Or does Kubernetes just use all of them (so all 100 nodes will have webapp component)? What if I want to assign certain component to certain nodes because those nodes have better machine specs (e.g. assign backend component to first 10 nodes with better spec)?

Thank you in advance.


Solution

  • So how does Kubernetes assign each webapp component to certain node?

    When a new pod comes up the kube-scheduler will find a fitting node based on certain criteria like requested/allocatable resources, node status, taints and tolerations, etc. There a tones of good blog posts on the details, like this one.

    What if I want to assign certain component to certain nodes because those nodes have better machine specs?

    There are different approaches to achieve that.

    The most basic one is a node selector. You just define a selector in the deployment manifest and the scheduler will find a node that matches the selector.

    There are also affinities. Those can be configured as node or pod affinities and as affinity or anti-affinity. By that you can basically tell the scheduler that it should try or has to find a node where for example no pod of the same deployment is running (that would be a PodAntiaffinity). I'd recommend reading the docs about this topic because it's quiet complex.

    There is also a relatively new concept, the topology spread constraint which let's you define how your pods should be spread throughout the cluster.

    The thing I don't get is that how do I make Kubernetes assign certain Service object (with associated Deployment object) to certain nodes?

    I put your initial question at the end because it's easier to understand the concept if you know how scheduling works.

    Every pod is (more or less) randomly assigned to one of your nodes. You can't know which one it's going to be but you know that the pods need to be reachable from outside the cluster and from other pods running inside the cluster.

    Each pod gets an ip assigned, but that's random too. Even if a pod reappears on the same node it's probably not going to get the same ip. You need a static object with a static ip so you can address each pod and also don't have to think about load balancing from client side.

    That's where services come in. They are virtual resources, meaning there is no process running somewhere that "is" the service. The service is rather a configuration that tells kube-proxy to set up routing on each node to a certain kind of pods (via selector).

    kube-proxy then configures (by default) iptables rules so requests for a certain ip (the one of the service) is routed to a pod. For convenience also dns is configured, so you can reach the service via <service-name.<namespace-name>, more about that in the docs.

    The benefit of the whole concept is that you don't have to think about any of the networking details, kubernetes does that for you. You define your deployment, where it's pods should run and how many and define a service so it can be addressed. Kubernetes does the heavy lifting of configuring and managing everything that is required to keep your application running in the desired state.