Search code examples
node.jsdockerkubernetesgoogle-cloud-platformmicroservices

Kubernetes NAT Streaming Server - Connection refused


I am working on a microservice web application (React and Node) using NATS and running it on GCP Kubernetes cluster.

I am unable to connect to the NAT streaming server pod. The ports 4222 & 8222 are open.I am getting connection refused error.

Error:

Using ts-node version 8.10.2, typescript version 3.9.7
NatsError: Could not connect to server: Error: connect ECONNREFUSED 10.32.13.69:4222
    at Socket.<anonymous> (/app/node_modules/nats/lib/nats.js:801:26)
    at Socket.emit (events.js:314:20)
    at emitErrorNT (internal/streams/destroy.js:100:8)
    at emitErrorCloseNT (internal/streams/destroy.js:68:3)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  code: 'CONN_ERR',
  chainedError: Error: connect ECONNREFUSED 10.32.13.69:4222
      at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1144:16) {
    errno: -111,
    code: 'ECONNREFUSED',
    syscall: 'connect',
    address: '10.32.13.69',
    port: 4222
  }
}

Pod + Service file for NAT Streaming Server

apiVersion: apps/v1
kind: Deployment 
metadata: 
    name: nats-depl
spec:
    replicas: 1
    selector: 
        matchLabels: 
            app: nats
    template:
        metadata: 
            labels: 
                app: nats
        spec:
            containers: 
                - name: nats 
                  image: nats-streaming:0.17.0 
                  ## args provides arguments to the primary
                  ## cmd executed when container starts 
                  args: [
                    '--port', '4222',
                    '--http_port', '8222',
                    '--hb_interval', '5s',      
                    '--hb_timeout', '5s',       
                    '--hb_fail_count', '2',     
                    '--stan_debug', 
                    '--cluster_id', 'ticketing'
                  ]

Service config :

---
apiVersion: v1
kind: Service 
metadata:
    name: nats-srv 
spec: 
    type: ClusterIP
    selector: 
        apps: nats 
    ports: 
        - name: client 
          protocol: TCP 
          port: 4222 
          targetPort: 4222 
        - name: monitoring 
          protocol: TCP 
          port: 8222 
          targetPort: 8222 

$ kubectl describe service nats-srv

$ kubectl describe service nats-srv
Name:              nats-srv
Namespace:         default
Labels:            app.kubernetes.io/managed-by=skaffold-v1.11.0
                   skaffold.dev/builder=google-cloud-build
                   skaffold.dev/cleanup=true
                   skaffold.dev/deployer=kubectl
                   skaffold.dev/run-id=59070209-9ab7-47cd-ab0f-cdd91797b1a7
                   skaffold.dev/tag-policy=git-commit
                   skaffold.dev/tail=true
Annotations:       kubectl.kubernetes.io/last-applied-configuration:
                     {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app.kubernetes.io/managed-by":"skaffold-v1.11.0","skaffold.dev...
Selector:          apps=nats
Type:              ClusterIP
IP:                10.32.13.69
Port:              client  4222/TCP
TargetPort:        4222/TCP
Endpoints:         <none>
Port:              monitoring  8222/TCP
TargetPort:        8222/TCP
Endpoints:         <none>
Session Affinity:  None
Events:            <none>


Solution

  • The service has selector apps: nats but deployment has label app: nats. They need to be same.This is why Endpoints in service is not having Pod IPs and hence you get connection refused. Modify the service as below

    ---
    apiVersion: v1
    kind: Service 
    metadata:
        name: nats-srv 
    spec: 
        type: ClusterIP
        selector: 
            app: nats 
        ports: 
            - name: client 
              protocol: TCP 
              port: 4222 
              targetPort: 4222 
            - name: monitoring 
              protocol: TCP 
              port: 8222 
              targetPort: 8222