Why Nginx ingress controller "steals" traffic from Apache running on host machine?

Question

I have a very interesting situation here. :-) I reinstalled my Kubernetes cluster (bare metal). I have 1 master and 2 worker nodes. On worker no 2, on the host machine, I have an Apache http server listening on port 80 and serving a web page.

NAME         STATUS   ROLES           AGE   VERSION
kubemaster   Ready    control-plane   22h   v1.26.3
kubenode-1   Ready    <none>          21h   v1.26.3
kubenode-2   Ready    <none>          21h   v1.26.3

I installed Nginx ingress controller on my cluster as a daemon set. So there are 2 controllers running on my 2 worker nodes.

NAME                READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
bkk-ingress-5p7b8   1/1     Running   0          31m   192.168.29.136   kubenode-1   <none>           <none>
bkk-ingress-rxjw4   1/1     Running   0          31m   192.168.77.72    kubenode-2   <none>           <none>

And here is the interesting part. When I want to visit the web page served by the Apache on the host machine the traffic is taken by the Nginx and I saw the Nginx ingress controller not found page instead of my web page. However only the apache is listening on port 80:

[root@kubenode-2 ~]$ netstat -tulpn | grep --color :80
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1099870/httpd
[root@kubenode-2 ~]$ netstat -tulpn | grep --color :443
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      1099870/httpd

The ingress service is a NodePort and it is not supposed to use port 80 or 443:

apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: bkk
    meta.helm.sh/release-namespace: nginx
  creationTimestamp: "2023-03-30T21:00:17Z"
  labels:
    app.kubernetes.io/instance: bkk
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: bkk-nginx-ingress
    helm.sh/chart: nginx-ingress-0.16.2
  name: bkk-nginx-ingress
  namespace: nginx
  resourceVersion: "159007"
  uid: 1f45c592-80e0-4fb5-acbd-a04c76772e6a
spec:
  clusterIP: 10.104.170.119
  clusterIPs:
  - 10.104.170.119
  externalTrafficPolicy: Local
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: http
    nodePort: 31902
    port: 801
    protocol: TCP
    targetPort: 801
  - name: https
    nodePort: 31903
    port: 4431
    protocol: TCP
    targetPort: 4431
  selector:
    app: bkk-ingress
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

And there is not a single ingress defined anywhere in my system:

[root@kubemaster helm-chart]$ kubectl get ingresses --all-namespaces
No resources found

Does anyone have any idea what is going on here? Why the Nginx, living and running as a Kubernetes pod, steals the traffic from the host machine?

Answer 1

I found it out... When I installed Nginx using Helm, it adds a hostPort:80 to the pod definition by default. Even it is not in the values.yaml, and not mentioned anywhere. So, if you remove that by hand, then Nginx won't steal port 80 from the Apache installed outside of Kubernetes.