Search code examples
nginxkuberneteslets-encrypt

How to use cert-manager with nginx ingress controller and avoid "host collusion" issue?


I recently updated the nginx-controller in my kubernetes cluster. The current behavior when multiple ingress resources are defined is "choose winner". Meaning that if there are 2 ingress resources to the same host, nginx will only define one of them, instead of merging the paths.

The way to get around this seems to be documented here, which describes the solution by adding annotations of nginx.org/mergeable-ingress-type.

However, to the best of my knowledge, cert-manager does not create the ingresses with these annotation. Resulting with a 404 for the challenge url (or worse: a 404 to the site I'm trying to create a certificate for)

Here is the ingress serving the site:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: farmers
  annotations:
    nginx.org/mergeable-ingress-type: "master"
spec:
  ingressClassName: nginx
  - host: farmers.klino.me
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: frontend
            port:
              number: 80

And the ingress of the challenge:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0,::/0
  generateName: cm-acme-http-solver-
  generation: 3
  labels:
    acme.cert-manager.io/http-domain: "..."
    acme.cert-manager.io/http-token: "..."
    acme.cert-manager.io/http01-solver: "true"
  name: cm-acme-http-solver-7677r
  namespace: staging
spec:
  rules:
  - host: farmers.klino.me
    http:
      paths:
      - backend:
          service:
            name: cm-acme-http-solver-hkx6g
            port:
              number: 8089
        path: /.well-known/acme-challenge/<...reducted...>
        pathType: ImplementationSpecific

In summary, I have 2 ingress resources to the host farmers.klino.me, on serving the site at /, the other - created automatically by certmanager - serving the challenge at /.well-known/acme-challenge/<...reducted...>.

How should I configure the nginx-ingress and cert-manager to merge the ingresses and work together without human intervention?


Solution

  • EDIT: There appears to be two different NGINX Ingress controller deployments. One called nginx-ingress and the other called ingress-nginx

    The one you want is ingress-nginx. This version automatically merges the ingress rules created by cert-manager and instantly all my certificates renewed. The controller documentation can be found here and the cert-manager documentation found here is applicable.

    Hope this helps!

    Original Post: I'm also experiencing the same issue. I've not yet tried it but it looks like you can configure an ingressTemplate for your given issuer. For example:

    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      name: ...
    spec:
      acme:
        server: ...
        privateKeySecretRef:
          name: ...
        solvers:
        - http01:
            ingress:
              ingressTemplate:
                metadata:
                  labels:
                    foo: "bar"
                  annotations:
                    "nginx.ingress.kubernetes.io/whitelist-source-range": "0.0.0.0/0,::/0"
                    "nginx.org/mergeable-ingress-type": "minion"
                    "traefik.ingress.kubernetes.io/frontend-entry-points": "http"
    

    See https://cert-manager.io/v1.6-docs/configuration/acme/http01/#ingresstemplate for more details.

    This doesn't seem like the most efficient way to handle this issue though so if anyone has a better solution I'm all ears.