Search code examples
kuberneteskubernetes-ingressnginx-ingressreadinessprobe

"Readiness probe failed: HTTP probe failed with statuscode: 503" at NGINX Insgress Controller pod in Kubernetes cluster


I wanted to expose a web application using NGINX Ingress Controller.

$ kubectl version
Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.2

NGINX Ingress controller version is 3.4.3

I searched that these versions are supposed to be compatible.

I followed the ingress controller installation on https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-manifests/ , choosing to install through manifests (since I am more used to them) and NodePort type of service.

$ kubectl apply -f deployments/common/ns-and-sa.yaml
$ kubectl apply -f deployments/rbac/rbac.yaml
$ kubectl apply -f deployments/common/nginx-config.yaml
$ kubectl apply -f deployments/common/ingress-class.yaml
$ kubectl apply -f deployments/deployment/nginx-ingress.yaml
$ kubectl create -f deployments/service/nodeport.yaml

After checking kubectl get all --all-namespaces I noticed the following on the nginx-ingress pod:

NAMESPACE              NAME                                             READY   STATUS    RESTARTS   AGE
nginx-ingress          pod/nginx-ingress-755bf8968b-tmdps               0/1     Running   0          76m

Afterwards I ran kubectl describe pod nginx-ingress-755bf8968b-tmdps -n nginx-ingress which showed:

Warning  Unhealthy  4m21s (x4549 over 79m)  kubelet  Readiness probe failed: HTTP probe failed with statuscode: 503

I also check the labels app=nginx-ingress on the pod and it's correct.

Outpuf of kubectl logs -l app=nginx-ingress -n nginx-ingress:

W0326 16:09:12.078133       1 reflector.go:539] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: failed to list *v1.VirtualServer: the server could not find the requested resource (get virtualservers.k8s.nginx.org)
E0326 16:09:12.078276       1 reflector.go:147] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: Failed to watch *v1.VirtualServer: failed to list *v1.VirtualServer: the server could not find the requested resource (get virtualservers.k8s.nginx.org)
W0326 16:09:32.474319       1 reflector.go:539] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: failed to list *v1.VirtualServerRoute: the server could not find the requested resource (get virtualserverroutes.k8s.nginx.org)
E0326 16:09:32.474345       1 reflector.go:147] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: Failed to watch *v1.VirtualServerRoute: failed to list *v1.VirtualServerRoute: the server could not find the requested resource (get virtualserverroutes.k8s.nginx.org)
W0326 16:09:39.265682       1 reflector.go:539] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: failed to list *v1.TransportServer: the server could not find the requested resource (get transportservers.k8s.nginx.org)
E0326 16:09:39.265849       1 reflector.go:147] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: Failed to watch *v1.TransportServer: failed to list *v1.TransportServer: the server could not find the requested resource (get transportservers.k8s.nginx.org)
W0326 16:09:48.080045       1 reflector.go:539] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: failed to list *v1.Policy: the server could not find the requested resource (get policies.k8s.nginx.org)
E0326 16:09:48.080101       1 reflector.go:147] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: Failed to watch *v1.Policy: failed to list *v1.Policy: the server could not find the requested resource (get policies.k8s.nginx.org)
W0326 16:09:57.631094       1 reflector.go:539] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: failed to list *v1.VirtualServer: the server could not find the requested resource (get virtualservers.k8s.nginx.org)
E0326 16:09:57.631121       1 reflector.go:147] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: Failed to watch *v1.VirtualServer: failed to list *v1.VirtualServer: the server could not find the requested resource (get virtualservers.k8s.nginx.org)

What can I do? Following tutorials and documentation there shouldn't be any complications with the installation.


Solution

  • What your logs indicate is that you are missing the Custom Resource Definitions (CRDs).

    W0326 16:09:12.078133       1 reflector.go:539] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: failed to list *v1.VirtualServer: the server could not find the requested resource (get virtualservers.k8s.nginx.org)
    

    The CRDs can be installed with the following commands.

    kubectl apply -f config/crd/bases/k8s.nginx.org_virtualservers.yaml
    kubectl apply -f config/crd/bases/k8s.nginx.org_virtualserverroutes.yaml
    kubectl apply -f config/crd/bases/k8s.nginx.org_transportservers.yaml
    kubectl apply -f config/crd/bases/k8s.nginx.org_policies.yaml
    kubectl apply -f config/crd/bases/k8s.nginx.org_globalconfigurations.yaml