NGINX Ingress with Helm - configure rate limiting response code

Question

I'm trying to set the limit-req-status-code for my nginx ingress, but I'm failing to do so. According to the docs, this setting belongs in a ConfigMap (as opposed to other rate limiting settings that are annotations). I created a configmap, but the setting doesn't get respected. How do I know? I've used fortio to run into the rate limit and it's still returning 503.

I've tried finding out how to name the map correctly and tried a lot of different names like suggested in these answers, no effect. I also tried to manually pass the configmap name to the helm call, no luck either. That's what I have right now:

ingress-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-ingress-config
data:
  limit-req-status-code: "429"

ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ include "fullname" . }}-ingress
  labels:
    app.kubernetes.io/name: {{ include "name" . }}
    app.kubernetes.io/instance: {{ .Release.Name }}
    app.kubernetes.io/version: "{{ .Release.Revision }}"
    app.kubernetes.io/managed-by: {{ .Release.Service }}
    helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
  annotations:
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/limit-rpm: "1000"
    nginx.ingress.kubernetes.io/limit-rps: "100"
spec:
  ingressClassName: "nginx-public"
  rules:
    - host: {{ .Values.ingress.host }}
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: {{ include "fullname" . }}-service
                port:
                  name: http

deploy call

helm upgrade ${{ inputs.release-name }} ${{ inputs.working-directory }} \
          --install \
          --namespace=${{ inputs.aws-role-name }} \
          --wait \
          --timeout=5m0s \
          --atomic \
          --values=${{ inputs.working-directory }}/values.yaml \
          --values=${{ inputs.working-directory }}/values-${{ inputs.stage-name }}.yaml \
          --set controller.config.name=nginx-ingress-config \
          --set-string deployment.image.registry="${{ secrets.mgmt-aws-account-id }}.dkr.ecr.${{ inputs.mgmt-aws-region }}.amazonaws.com" \
          --set-string deployment.image.repository="${{ inputs.image-name }}" \
          --set-string deployment.image.digest="${{ inputs.image-digest }}" \
          --set-string database.user='${{ steps.fetch-secret-postgres-username.outputs.aws-secret-value }}' \
          --set-string database.password='${{ steps.fetch-secret-postgres-password.outputs.aws-secret-value }}'

What am I doing wrong?

Answer 1

Ok so I found a solution in another question: I can pass the setting as a configuration snippet in an annotation:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ include "fullname" . }}-ingress
  labels:
    app.kubernetes.io/name: {{ include "name" . }}
    app.kubernetes.io/instance: {{ .Release.Name }}
    app.kubernetes.io/version: "{{ .Release.Revision }}"
    app.kubernetes.io/managed-by: {{ .Release.Service }}
    helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
  annotations:
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/limit-rpm: "{{ .Values.ingress.requestsPerMinute }}"
    nginx.ingress.kubernetes.io/limit-rps: "{{ .Values.ingress.requestsPerSecond }}"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      limit_req_status 429;                                      <<<< this is the important one
spec:
  ingressClassName: "nginx-public"
  rules:
    - host: {{ .Values.ingress.host }}
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: {{ include "fullname" . }}-service
                port:
                  name: http

The configmap and the configured controller.config.name are not necessary.