Search code examples
kubernetesload-balancingstatefulset

How can I expose a Statefulset with a load balancer?


I currently trying to create a cluster of X pods witch each have a personal persistent volume. To do that I've created a StateFulSet with X replicas and a PersistentVolumeClaimTemplate This part is working.

The problem is that it's seem's to be impossible to expose thoses pods with a LoadBalancer in the same way as a deployment (because of the uniqueness of a pods in a statefulset).

At this moment I've tried to expose it as a simple deployment witch is not working and the only way I've found is to expose each pods one by one (I've not tested it but I saw it on this) but it's not that scalable...

I'm not running kubernetes on any cloud provider platform then please avoid exclusive command line.


Solution

  • The problem is that it's seem's to be impossible to expose thoses pods with a LoadBalancer in the same way as a deployment (because of the uniqueness of a pods in a statefulset).

    Why not? Here is my StatefulSet with default Nginx

    $ k -n test get statefulset
    NAME      DESIRED   CURRENT   AGE
    web       2         2         5d
    $ k -n test get pods
    web-0                    1/1       Running   0          5d
    web-1                    1/1       Running   0          5d
    

    Here is my Service type LoadBalancer which is NodePort (in fact) in case of Minikube

    $ k -n test get svc
    NAME      TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
    nginx     LoadBalancer   10.110.22.74   <pending>     80:32710/TCP   5d
    

    Let's run some pod with curl and do some requests to ClusterIP:

    $ kubectl -n test run -i --tty tools --image=ellerbrock/alpine-bash-curl-ssl -- bash
    bash-4.4$ curl 10.110.22.74 &> /dev/null
    bash-4.4$ curl 10.110.22.74 &> /dev/null
    bash-4.4$ curl 10.110.22.74 &> /dev/null
    bash-4.4$ curl 10.110.22.74 &> /dev/null
    

    Let's check out Nginx logs:

    $ k -n test logs web-0
    172.17.0.7 - - [18/Apr/2019:23:35:04 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.61.0"
    172.17.0.7 - - [18/Apr/2019:23:35:05 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.61.0"
    172.17.0.7 - - [18/Apr/2019:23:35:17 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.61.0"
    $ k -n test logs web-1
    172.17.0.7 - - [18/Apr/2019:23:35:15 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.61.0"
    

    172.17.0.7 - is my pod with curl:

    NAME                     READY     STATUS    RESTARTS   AGE       IP           NODE
    tools-654cfc5cdc-8zttt   1/1       Running   1          5d        172.17.0.7   minikube
    

    Actually ClusterIP is totally enough in case of load balancing between StatefulSet's pods, because you have a list of Endpoints

    $ k -n test get endpoints
    NAME      ENDPOINTS                     AGE
    nginx     172.17.0.5:80,172.17.0.6:80   5d
    

    YAMLs:

    apiVersion: apps/v1beta1
    kind: StatefulSet
    metadata:
      name: web
    spec:
      serviceName: "nginx"
      replicas: 2
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: k8s.gcr.io/nginx-slim:0.8
            ports:
            - containerPort: 80
              name: web
    

    apiVersion: v1
    kind: Service
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      type: LoadBalancer
      ports:
      - port: 80
        name: web
      selector:
        app: nginx