Load is not balanced with Kubernetes Services

Question

I created two replicas of nginx with following yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.20-alpine
          ports:
            - containerPort: 80

And I created service with:

apiVersion: v1
kind: Service
metadata:
  name: nginx-test-service
spec:
  selector:
    app: nginx
  ports:
    - port: 8082
      targetPort: 80

Everything looks good. But when I do

minikube service nginx-test-service

I am able to access the nginx. But when I see the two pods logs, the request is always going to single pod. The other pod is not getting any request.

But, kubernetes service should do the load balancing right?

Am I missing anything?

Answer 1

One way to get load balancing on-premise running is with ip virtual services. (ipvs). It;s a service which hands out ip's of the next pod to schedule/call

it's likely installed already.

lsmod | grep ip_vs
ip_vs_sh               16384  0
ip_vs_wrr              16384  0
ip_vs_rr               16384  19

Have your cni properly setup and run

kubectl edit cm -n kube-system kube-proxy

edit the ipvs section

set mode to ipvs

mode: "ipvs"

and the ipvs section

ipvs:
      excludeCIDRs: null
      minSyncPeriod: 0s
      scheduler: "rr"

As always there are lots of variables biting each other with k8s, but it is possible with ipvs.

https://kubernetes.io/blog/2018/07/09/ipvs-based-in-cluster-load-balancing-deep-dive/