On an EKS Kubernetes 1.21 cluster, multiple new pods fail to become deployed and remain in pending status.
I'm getting this event in my pods: pod has unbound immediate PersistentVolumeClaims
The affected pods are supposed to mount PVCs/PVs which use AWS EFS. How can I find out what's wrong?
This may be due to an EFS Access point limit, which is currently set to 120.
You should be able to see this in the logs of efs-csi-controller for container csi-provisioner:
ErrorCode: "AccessPointLimitExceeded",
Message_: "You have reached the maximum number of access points (120) for your file system fs-(...). Delete an access point and add a new one."
And it should also be visible in the events of the affected PVC:
Warning ProvisioningFailed (...) failed to provision volume with StorageClass "(...)": rpc error: code = Internal desc = Failed to create Access point in File System fs-(...) : Failed to create access point: AccessPointLimitExceeded: You have reached the maximum number of access points (120) for your file system fs-(...). Delete an access point and add a new one.
The solution is therefore to either delete existing PVs and their underlying EFS access points(!). Or create a new EFS drive for future PVs and PVCs.
When deleting PVs in order to delete access points, keep in mind that deleting a PV with reclaim policy Retain will not trigger deletion of its underlying access point.