Search code examples
reactjskubernetescluster-computingnode-jose

How to connect front to back in k8s cluster internal (connection refused)


Error while trying to connect React frontend web to nodejs express api server into kubernetes cluster.

Can navigate in browser to http:localhost:3000 and web site is ok.

But can't navigate to http:localhost:3008 as expected (should not be exposed)

My goal is to pass REACT_APP_API_URL environment variable to frontend in order to set axios baseURL and be able to establish communication between front and it's api server.

deploy-front.yml

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: gbpd-front
spec:
  selector:
    matchLabels:
      app: gbpd-api
      tier: frontend
      track: stable
  replicas: 2
  template:
    metadata:
      labels:
        app: gbpd-api
        tier: frontend
        track: stable
    spec:
      containers:
        - name: react
          image: binomio/gbpd-front:k8s-3
          ports:
            - containerPort: 3000
          resources:
            limits:
              memory: "150Mi"
            requests:
              memory: "100Mi"
          imagePullPolicy: Always

service-front.yaml

apiVersion: v1
kind: Service
metadata:
  name: gbpd-front
spec:
  selector:
    app: gbpd-api
    tier: frontend
  ports:
  - protocol: "TCP"
    port: 3000
    targetPort: 3000
  type: LoadBalancer

Deploy-back.yaml

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: gbpd-api
spec:
  selector:
    matchLabels:
      app: gbpd-api
      tier: backend
      track: stable
  replicas: 3 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: gbpd-api
        tier: backend
        track: stable
    spec:
      containers:
        - name: gbpd-api
          image: binomio/gbpd-back:dev
          ports:
            - name: http
              containerPort: 3008

service-back.yaml

apiVersion: v1
kind: Service
metadata:
  name: gbpd-api
spec:
  selector:
    app: gbpd-api
    tier: backend
  ports:
  - protocol: "TCP"
    port: 3008
    targetPort: http

I tried many combinations, also tried adding "LoadBalancer" to backservice but nothing...

I can connect perfecto to localhost:3000 and use frontend but frontend can't connect to backend service.

Question 1: What's is the ip/name to use in order to pass REACT_APP_API_URL to fronten correctly? Question 2: Why is curl localhost:3008 not answering?

After 2 days trying almost everything in k8s official docs... can't figure out what's happening here, so any help will be much appreciated.

kubectl describe svc gbpd-api Response:

kubectl describe svc gbpd-api
Name:                     gbpd-api
Namespace:                default
Labels:                   <none>
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"gbpd-api","namespace":"default"},"spec":{"ports":[{"port":3008,"p...
Selector:                 app=gbpd-api,tier=backend
Type:                     LoadBalancer
IP:                       10.107.145.227
LoadBalancer Ingress:     localhost
Port:                     <unset>  3008/TCP
TargetPort:               http/TCP
NodePort:                 <unset>  31464/TCP
Endpoints:                10.1.1.48:3008,10.1.1.49:3008,10.1.1.50:3008
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

Solution

  • I tested your environment, and it worked when using a Nginx image, let's review the environment:

    • The front-deployment is correctly described.
    • The front-service exposes it as loadbalancer, meaning your frontend is accessible from outside, perfect.
    • The back deployment is also correctly described.
    • The backend-service stays with as ClusterIP in order to be only accessible from inside the cluster, great.

    Below I'll demonstrate the communication between front-end and back end.

    • I'm using the same yamls you provided, just changed the image to Nginx for example purposes, and since it's a http server I'm changing containerport to 80.

    Question 1: What's is the ip/name to use in order to pass REACT_APP_API_URL to fronten correctly?

    • I added the ENV variable to the front deploy as requested, and I'll use it to demonstrate also. You must use the service name to curl, I used the short version because we are working in the same namespace. you can also use the full name: http://gbpd-api.default.svc.cluster.local:3008

    Reproduction:

    • Create the yamls and applied them:
    $ cat deploy-front.yaml 
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: gbpd-front
    spec:
      selector:
        matchLabels:
          app: gbpd-api
          tier: frontend
          track: stable
      replicas: 2
      template:
        metadata:
          labels:
            app: gbpd-api
            tier: frontend
            track: stable
        spec:
          containers:
            - name: react
              image: nginx
              env:
                - name: REACT_APP_API_URL
                  value: http://gbpd-api:3008
              ports:
                - containerPort: 80
              resources:
                limits:
                  memory: "150Mi"
                requests:
                  memory: "100Mi"
              imagePullPolicy: Always
    
    $ cat service-front.yaml 
    cat: cat: No such file or directory
    apiVersion: v1
    kind: Service
    metadata:
      name: gbpd-front
    spec:
      selector:
        app: gbpd-api
        tier: frontend
      ports:
      - protocol: "TCP"
        port: 3000
        targetPort: 80
      type: LoadBalancer
    
    $ cat deploy-back.yaml 
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: gbpd-api
    spec:
      selector:
        matchLabels:
          app: gbpd-api
          tier: backend
          track: stable
      replicas: 3
      template:
        metadata:
          labels:
            app: gbpd-api
            tier: backend
            track: stable
        spec:
          containers:
            - name: gbpd-api
              image: nginx
              ports:
                - name: http
                  containerPort: 80
    
    $ cat service-back.yaml 
    apiVersion: v1
    kind: Service
    metadata:
      name: gbpd-api
    spec:
      selector:
        app: gbpd-api
        tier: backend
      ports:
      - protocol: "TCP"
        port: 3008
        targetPort: http
    
    $ kubectl apply -f deploy-front.yaml 
    deployment.apps/gbpd-front created
    $ kubectl apply -f service-front.yaml 
    service/gbpd-front created
    $ kubectl apply -f deploy-back.yaml 
    deployment.apps/gbpd-api created
    $ kubectl apply -f service-back.yaml 
    service/gbpd-api created
    
    • Remember, in Kubernetes the communication is designed to be made between services, because the pods are always recreated when there is a change in the deployment or when the pod fail.
    $ kubectl get all
    NAME                              READY   STATUS    RESTARTS   AGE
    pod/gbpd-api-dc5b4b74b-kktb9      1/1     Running   0          41m
    pod/gbpd-api-dc5b4b74b-mzpbg      1/1     Running   0          41m
    pod/gbpd-api-dc5b4b74b-t6qxh      1/1     Running   0          41m
    pod/gbpd-front-66b48f8b7c-4zstv   1/1     Running   0          30m
    pod/gbpd-front-66b48f8b7c-h58ds   1/1     Running   0          31m
    
    NAME                 TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)          AGE
    service/gbpd-api     ClusterIP      10.0.10.166   <none>         3008/TCP         40m
    service/gbpd-front   LoadBalancer   10.0.11.78    35.223.4.218   3000:32411/TCP   42m
    
    • The pods are the workers, and since they are replaceable by nature, we will connect to a frontend pod to simulate his behaviour and try to connect to the backend service (which is the network layer that will direct the traffic to one of the backend pods).
    • The nginx image does not come with curl preinstalled, so I will have to install it for demonstration purposes:
    $ kubectl exec -it pod/gbpd-front-66b48f8b7c-4zstv -- /bin/bash
    root@gbpd-front-66b48f8b7c-4zstv:/# apt update && apt install curl -y
    done.
    
    root@gbpd-front-66b48f8b7c-4zstv:/# curl gbpd-api:3008
    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    ...
    
    • Now let's try using the environment variable that was defined:
    root@gbpd-front-66b48f8b7c-4zstv:/# printenv | grep REACT
    REACT_APP_API_URL=http://gbpd-api:3008
    root@gbpd-front-66b48f8b7c-4zstv:/# curl $REACT_APP_API_URL
    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    ...
    

    Considerations:

    Question 2: Why is curl localhost:3008 not answering?

    • Since all yamls are correctly described you must check if image: binomio/gbpd-back:dev is correctly serving on port 3008 as intended.
    • Since it's not a public image, I can't test it, so I'll give you troubleshooting steps:
      • just like we logged inside the front-end pod you will have to log into this backend-pod and test curl localhost:3008.
      • If it's based on a linux distro with apt-get, you can run the commands just like I did on my demo:
      • get the pod name from backend deploy (example: gbpd-api-6676c7695c-6bs5n)
      • run kubectl exec -it pod/<POD_NAME> -- /bin/bash
      • then run apt update && apt install curl -y
      • and test curl localhost:3008
      • if no answer run `apt update && apt install net-tools
      • and test netstat -nlpt, it will have to show you the output of the services running and the respective port, example:
    root@gbpd-api-585df9cb4d-xr6nk:/# netstat -nlpt
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
    tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1/nginx: master pro 
    
    • If the pod does not return nothing even on this approach, you will have to check the code in the image.

    Let me know if you need help after that!