Access a web service in kubernetes pod from local browser using NodePort yields Connection refused

Question

What do I need to do in order to get my local browser to and request a resource to a web service running inside Minikube instance running locally on my machine?

I am getting a Connection refused when trying to kubectl port-forward.

My workflow is:

1. Creating Dockerfile with web service on
2. Start minikube in docker
3. Build docker image
4. Import image locally into Minikube
5. Created a deployment with one container and a NodePort service
6. Applied deployment/service
7. Ran kubectl port-forward (to hopefully forward requests to my container)
8. Open browser to 127.0.0.1:31000

Port Configuration Summary

• Dockerfile:
  • Expose: 80
  • uvicorn: 80
• Deployment
  • NodePort Service:
    • Port: 80
    • Target Port: 80
    • Node Port: 31000
• Kubectl Command: 8500:31000
• Browser: 127.0.0.1:8500

Setup and run through

dev.dockerfile (Step 1)

FROM python:3.11-buster # Some Debian Python image...  I built my own

COPY ../sources/api/ /app/
RUN pip install --no-cache-dir --upgrade -r /app/requirements.txt

ENV PYTHONPATH=/app/

EXPOSE 80

CMD ["uvicorn", "app.main:app", "--proxy-headers", "--host", "0.0.0.0", "--port", "80"]

Build Sequence (Steps 2 to 4)

# 2 - start minikube
minikube start --bootstrapper=kubeadm --vm-driver=docker
minikube docker-env

## 3 - build image
docker build -f ../../service1/deploy/dev.dockerfile ../../service1 -t acme-app.service1:latest 

## 4 - load image into minikube
minikube image load acme-app.service1:latest

Deployment (Step 5 and 6)

deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: acme-service-1-deployment
  namespace: acme-app-dev
  labels:
    app: service-1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: service-1
  template:
    metadata:
      labels:
        app: service-1
    spec:
      containers:
        - name: service1-container
          image: docker.io/library/acme-app.service1:latest
          imagePullPolicy: Never
          ports:
          - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: service-1-service
  namespace: acme-app-dev
spec:
  type: NodePort
  selector:
    app: service-1
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 31000

Deploy

kubectl apply -f deployment.yaml

kubectl port forward (Step 7)

Find Pod

kubectl get pods -n acme-app-dev
NAME                                         READY   STATUS    RESTARTS   AGE
acme-service-1-deployment-76748d7ff6-llcsr   1/1     Running   0          11s

Port Forward to pod

port-forward acme-service-1-deployment-76748d7ff6-llcsr 8500:31000 -n acme-app-dev
Forwarding from 127.0.0.1:8500 -> 31000
Forwarding from [::1]:8500 -> 31000

Test in Browser (Step 8)

Open favorite browser and navigate to 127.0.0.1:31000.

The console running the port forward now outputs:

E0123 14:54:16.208010   25932 portforward.go:406] an error occurred forwarding 8500 -> 31000: error forwarding port 31000 to pod d4c0fa6cb16ce02335a05cad904fbf2ab7818e2073d7c7ded8ad05f193aa37e7, uid : exit status 1: 2023/01/23 14:54:16 socat[39370] E connect(5, AF=2 127.0.0.1:31000, 16): Connection refused
E0123 14:54:16.213268   25932 portforward.go:234] lost connection to pod

What have I looked at?

I've tried looking through the docs on kubernetes website as well as issues on here (yes there are similar). This is pretty similar - although no marked answer and still an issue by the looks of it. I couldn't see a solution for my issue here.

NodePort exposed Port connection refused

I am running Minikube on Windows and I'm just setting out on a kubernetes journey.

The image itself works in docker from a docker compose. I can see the pod is up and running in minikube from the logs (minikube dashboard).

Answer 1

You got your wires crossed:

• The pod is listening on port 80
• The NodePort service is listening on port 31000 on the node, but its underlying ClusterIP service is listening on port 80 as well.
• You are trying to port-forward to port 31000 on the Pod. This will not work.

Call one of the following instead:

• kubectl port-forward -n acme-app-dev deploy/acme-service-1-deployment 8500:80
• or kubectl port-forward -n acme-app-dev service/service-1-service 8500:80
• or use minikube service -n acme-app-dev service-1-service and use the provided URL.