Search code examples
kubernetesmulti-tenantminikubedistributed-system

Testing locally k8s distributed system


I'm new to k8s and I'm trying to build a distributed system. The idea is that a stateful pod will be spawened for each user.

Main services are two Python applications MothershipService and Ship. MothershipService's purpose is to keep track of ship-per-user, do health checks, etc. Ship is running some (untrusted) user code.

MothershipService           Ship-user1
|              | ---------- |        |---vol1
|..............| -----.     |--------|
                       \    
                        \   Ship-user2
                         '- |        |---vol2
                            |--------|

I can manage fine to get up the ship service

> kubectl get all -o wide
NAME           READY   STATUS    RESTARTS   AGE   IP            NODE       NOMINATED NODE   READINESS GATES
pod/ship-0     1/1     Running   0          7d    10.244.0.91   minikube   <none>           <none>

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE   SELECTOR
service/ship         ClusterIP   None         <none>        8000/TCP   7d    app=ship
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP    7d    <none>

NAME                      READY   AGE   CONTAINERS   IMAGES
statefulset.apps/ship     1/1     7d    ship       ship

My question is how do I go about testing this via curl or a browser? These are all backend services so NodePort seems not the right approach since none of this should be accessible to the public. Eventually I will build a test-suite for all this and deploy on GKE.

ship.yml (pseudo-spec)

kind: Service
metadata:
  name: ship
spec:
  ports:
  - port: 8000
    name: ship
  clusterIP: None  # headless service
  ..

---

kind: StatefulSet
metadata:
  name: ship
spec:
  serviceName: "ship"
  replicas: 1
  template:
    spec:
      containers:
      - name: ship
        image: ship
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8000
          name: ship
  ..

Solution

  • One possibility is to use the kubectl port-forward command to expose the pod port locally on your system. For example, if I'm use this deployment to run a simple web server listening on port 8000:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: example
      name: example
    spec:
      selector:
        matchLabels:
          app: example
      template:
        metadata:
          labels:
            app: example
        spec:
          containers:
          - args:
            - --port
            - "8000"
            image: docker.io/alpinelinux/darkhttpd
            name: web
            ports:
            - containerPort: 8000
              name: http
    

    I can expose that on my local system by running:

    kubectl port-forward deploy/example 8000:8000
    

    As long as that port-forward command is running, I can point my browser (or curl) at http://localhost:8000 to access the service.


    Alternately, I can use kubectl exec to run commands (like curl or wget) inside the pod:

    kubectl exec -it web -- wget -O- http://127.0.0.1:8000