Search code examples
kuberneteskubernetes-ingressnginx-ingress

how to fix ingress-nginx say return 404


I'm trying to learn nginx-ingress-controller per https://devopscube.com/setup-ingress-kubernetes-nginx-controller

my laptop is at 192.168.12.71

my ingress-nginx-controller pod and service is at 192.168.1.67.

a. Pod

 bino@corobalap  ~/k8nan/ingresnginx/nginx-ingress-controller   bino01 ±  kubectl --namespace ingress-nginx describe pod ingress-nginx-controller
Name:         ingress-nginx-controller-78f456c879-w6pbd
Namespace:    ingress-nginx
Priority:     0
Node:         bino-k8-wnode1/192.168.1.67
Start Time:   Tue, 19 Jul 2022 12:49:19 +0700
Labels:       app.kubernetes.io/component=controller
              app.kubernetes.io/instance=ingress-nginx
              app.kubernetes.io/name=ingress-nginx
              pod-template-hash=78f456c879
Annotations:  <none>
Status:       Running
IP:           192.168.1.67
IPs:
  IP:           192.168.1.67
Controlled By:  ReplicaSet/ingress-nginx-controller-78f456c879
Containers:
  controller:
    Container ID:  containerd://8606c2dd3800502eb56dd6de2decab93f2b9567916b462a83acf8917bcb7696d
    Image:         k8s.gcr.io/ingress-nginx/controller:v1.1.1
    Image ID:      k8s.gcr.io/ingress-nginx/controller@sha256:0bc88eb15f9e7f84e8e56c14fa5735aaa488b840983f87bd79b1054190e660de
    Ports:         80/TCP, 443/TCP, 8443/TCP
    Host Ports:    80/TCP, 443/TCP, 8443/TCP
    Args:
      /nginx-ingress-controller
      --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
      --election-id=ingress-controller-leader
      --controller-class=k8s.io/ingress-nginx
      --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
      --validating-webhook=:8443
      --validating-webhook-certificate=/usr/local/certificates/cert
      --validating-webhook-key=/usr/local/certificates/key
    State:          Running
      Started:      Tue, 19 Jul 2022 12:49:21 +0700
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:      100m
      memory:   90Mi
    Liveness:   http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=5
    Readiness:  http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
    Environment:
      POD_NAME:       ingress-nginx-controller-78f456c879-w6pbd (v1:metadata.name)
      POD_NAMESPACE:  ingress-nginx (v1:metadata.namespace)
      LD_PRELOAD:     /usr/local/lib/libmimalloc.so
    Mounts:
      /usr/local/certificates/ from webhook-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5tzs5 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  webhook-cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  ingress-nginx-admission
    Optional:    false
  kube-api-access-5tzs5:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

b. service :

 ✘ bino@corobalap  ~/k8nan/ingresnginx/nginx-ingress-controller   bino01 ±  kubectl --namespace ingress-nginx describe service ingress-nginx-controller
Name:                     ingress-nginx-controller
Namespace:                ingress-nginx
Labels:                   app.kubernetes.io/component=controller
                          app.kubernetes.io/instance=ingress-nginx
                          app.kubernetes.io/name=ingress-nginx
Annotations:              <none>
Selector:                 app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.109.72.34
IPs:                      10.109.72.34
Port:                     http  80/TCP
TargetPort:               http/TCP
NodePort:                 http  31324/TCP
Endpoints:                192.168.1.67:80
Port:                     https  443/TCP
TargetPort:               https/TCP
NodePort:                 https  31645/TCP
Endpoints:                192.168.1.67:443
Session Affinity:         None
External Traffic Policy:  Local
Events:                   <none>

my hello-app pod and service and ingress object also at 192.168.1.67

a. pod :

 bino@corobalap  ~/k8nan/ingresnginx/nginx-ingress-controller   bino01 ±  kubectl --namespace dev describe pod hello-app
Name:         hello-app-5c554f556c-4jhhv
Namespace:    dev
Priority:     0
Node:         bino-k8-wnode1/192.168.1.67
Start Time:   Tue, 19 Jul 2022 11:45:09 +0700
Labels:       app=hello
              pod-template-hash=5c554f556c
Annotations:  <none>
Status:       Running
IP:           10.244.1.19
IPs:
  IP:           10.244.1.19
Controlled By:  ReplicaSet/hello-app-5c554f556c
Containers:
  hello:
    Container ID:   containerd://8f4c02f60c82a70db4f7d0954ee19f606493a6ee5517d0d0f7641429682d86fb
    Image:          gcr.io/google-samples/hello-app:2.0
    Image ID:       gcr.io/google-samples/hello-app@sha256:2b0febe1b9bd01739999853380b1a939e8102fd0dc5e2ff1fc6892c4557d52b9
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 19 Jul 2022 11:45:16 +0700
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-6zghn (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-6zghn:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

Name:         hello-app-5c554f556c-c879k
Namespace:    dev
Priority:     0
Node:         bino-k8-wnode1/192.168.1.67
Start Time:   Tue, 19 Jul 2022 11:45:09 +0700
Labels:       app=hello
              pod-template-hash=5c554f556c
Annotations:  <none>
Status:       Running
IP:           10.244.1.21
IPs:
  IP:           10.244.1.21
Controlled By:  ReplicaSet/hello-app-5c554f556c
Containers:
  hello:
    Container ID:   containerd://37ff6a278c3e5398ff3c70d1b2db47bfaa421a757eefaf4f34befa83b9fd8569
    Image:          gcr.io/google-samples/hello-app:2.0
    Image ID:       gcr.io/google-samples/hello-app@sha256:2b0febe1b9bd01739999853380b1a939e8102fd0dc5e2ff1fc6892c4557d52b9
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 19 Jul 2022 11:45:17 +0700
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-xr2hw (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-xr2hw:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

Name:         hello-app-5c554f556c-n8pf8
Namespace:    dev
Priority:     0
Node:         bino-k8-wnode1/192.168.1.67
Start Time:   Tue, 19 Jul 2022 11:45:09 +0700
Labels:       app=hello
              pod-template-hash=5c554f556c
Annotations:  <none>
Status:       Running
IP:           10.244.1.20
IPs:
  IP:           10.244.1.20
Controlled By:  ReplicaSet/hello-app-5c554f556c
Containers:
  hello:
    Container ID:   containerd://833ac8b81b454261b472cf8e2d790cdf712a2e038074acd8d53563c25c677bdd
    Image:          gcr.io/google-samples/hello-app:2.0
    Image ID:       gcr.io/google-samples/hello-app@sha256:2b0febe1b9bd01739999853380b1a939e8102fd0dc5e2ff1fc6892c4557d52b9
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 19 Jul 2022 11:45:17 +0700
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-2hh5w (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-2hh5w:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

b. service

 bino@corobalap  ~/k8nan/ingresnginx/nginx-ingress-controller   bino01 ±  kubectl --namespace dev describe service hello-service
Name:              hello-service
Namespace:         dev
Labels:            app=hello
Annotations:       <none>
Selector:          app=hello
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.97.83.176
IPs:               10.97.83.176
Port:              <unset>  80/TCP
TargetPort:        8080/TCP
Endpoints:         10.244.1.19:8080,10.244.1.20:8080,10.244.1.21:8080
Session Affinity:  None
Events:            <none>

c. ingress object

 ✘ bino@corobalap  ~/k8nan/ingresnginx/nginx-ingress-controller   bino01 ±  kubectl --namespace dev describe ingress hello-app-ingress
Name:             hello-app-ingress
Labels:           <none>
Namespace:        dev
Address:          
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host        Path  Backends
  ----        ----  --------
  kopet.kpt   
              /   hello-service:80 (10.244.1.19:8080,10.244.1.20:8080,10.244.1.21:8080)
Annotations:  <none>
Events:       <none>

I tried to access hello-service localy at node where it life, got:

ubuntu@bino-k8-wnode1:~$ curl http://10.244.1.19:8080
Hello, world!
Version: 2.0.0
Hostname: hello-app-5c554f556c-4jhhv

I set a dummy hostname 'kopet.kpt' to point to the node where all the things life. but when i tried to curl from my laptop, nginx say 404

 bino@corobalap  ~/k8nan/ingresnginx/nginx-ingress-controller   bino01 ±  curl http://kopet.kpt/
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

Kindly please tell me what todo/read to fix it

-bino-


Solution

  • Your ingress definition created rules that proxy traffic from one path to another. In your case I believe the reason it's not working is that the app is proxied to app-service:80/app but your intending on serving traffic at the /root. Please try adding this annotation to your ingress resource:

    nginx.ingress.kubernetes.io/rewrite-target: / Refer github link for more information.

    Also, please provide a complete error message.