we have an EKS cluster on 1.21.
There is an nginx-ingress-controller-default-ingress-controller deployed with a Classic Load Balancer.
Suddenly, its pods are crashing with following errors.
I0815 04:40:04.970835 8 flags.go:204] Watching for Ingress class: nginx
W0815 04:40:04.980149 8 flags.go:249] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
W0815 04:40:04.980218 8 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
W0815 04:40:04.980255 8 client_config.go:548] error creating inClusterConfig, falling back to default config: open /var/run/secrets/kubernetes.io/serviceaccount/token: permission denied
F0815 04:40:04.980417 8 main.go:272] Error while initiating a connection to the Kubernetes API server. This could mean the cluster is misconfigured (e.g. it has invalid API server certificates or Service Accounts configuration). Reason: open /var/run/secrets/kubernetes.io/serviceaccount/token: permission denied
Below are the securityContext and VolumeMount of the pod.
securityContext:
allowPrivilegeEscalation: true
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
runAsUser: 101
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-k7n9n
readOnly: true
I tried to change runAsUser : 0, but it throws back with message the pods ".." is invalid.
Can you please give me some directions on what can be wrong here and any possible solution?
Try adding fsGroup. This will make serviceaccount directory readable by non-root user:
spec:
template:
spec:
securityContext:
fsGroup: 65534