Search code examples
google-kubernetes-enginequota

GKE - Quotas on Ingress objects


It looks there is a quota enforced by GKE on the number of Ingress objects (max 100). I don't see this limit documented anywhere, has anyone seen it before and understands where it comes from ? The error message returned is:

Error from server (Forbidden): error when creating "myweb.yaml": ingresses.networking.k8s.io "my-ingress" is forbidden: exceeded quota: gke-resource-quotas, requested: count/ingresses.networking.k8s.io=1, used: count/ingresses.networking.k8s.io=100, limited: count/ingresses.networking.k8s.io=100

Thanks in advance.


Solution

  • As documented here, Google automatically applies a set of resource quotas to clusters with ten nodes or fewer and to namespaces on those clusters. You can check the resources quotas for your cluster by running the following command:

    kubectl get resourcequota gke-resource-quotas -o yaml
    

    I created a new cluster with 3 nodes and here's the result of the above command on that cluster:

    apiVersion: v1
    kind: ResourceQuota
    metadata:
      creationTimestamp: "2021-01-07T16:12:00Z"
      name: gke-resource-quotas
      namespace: default
      resourceVersion: "1300"
      selfLink: /api/v1/namespaces/default/resourcequotas/gke-resource-quotas
      uid: <redacted>
    spec:
      hard:
        count/ingresses.extensions: "100"
        count/jobs.batch: 5k
        pods: "1500"
        services: "500"
    status:
      hard:
        count/ingresses.extensions: "100"
        count/jobs.batch: 5k
        pods: "1500"
        services: "500"
      used:
        count/ingresses.extensions: "0"
        count/jobs.batch: "0"
        pods: "0"
        services: "1"
    

    When I created a cluster with 11 nodes, this command returned resourcequotas "gke-resource-quotas" not found meaning that this quota is indeed applied to cluster with 10 nodes or less.