Search code examples
kuberneteskubernetes-ingresslets-encryptacme

Certificate always in 'False' state using LetsEncrypt with cluster issuer in k8s


I am unable to issue a working certificate for my ingress host in k8s. I use a ClusterIssuer to issue certificates and the same ClusterIssuer has issued certificates in the past for my ingress hosts under my domain name *xyz.com. But all of a sudden neither i can issue new Certificate with state 'True' for my host names nor a proper certificate secret (kubernetes.io/tls) gets created (but instead an Opaque secret gets created).


**strong text**

**kubectl describe certificate ingress-cert -n abc**

Name:         ingress-cert
Namespace:    abc
Labels:       <none>
Annotations:  <none>
API Version:  cert-manager.io/v1beta1
Kind:         Certificate
Metadata:
  Creation Timestamp:  2021-09-08T07:48:32Z
  Generation:          1
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  test-ingress
    UID:                   c03ffec0-df4f-4dbb-8efe-4f3550b9dcc1
  Resource Version:        146643826
  Self Link:               /apis/cert-manager.io/v1beta1/namespaces/abc/certificates/ingress-cert
  UID:                     90905ab7-22d2-458c-b956-7100c4c77a8d
Spec:
  Dns Names:
    abc.xyz.com
  Issuer Ref:
    Group:      cert-manager.io
    Kind:       ClusterIssuer
    Name:       letsencrypt
  Secret Name:  ingress-cert
Status:
  Conditions:
    Last Transition Time:        2021-09-08T07:48:33Z
    Message:                     Issuing certificate as Secret does not exist
    Reason:                      DoesNotExist
    Status:                      False
    Type:                        Ready
    Last Transition Time:        2021-09-08T07:48:33Z
    Message:                     Issuing certificate as Secret does not exist
    Reason:                      DoesNotExist
    Status:                      True
    Type:                        Issuing
  Next Private Key Secret Name:  ingress-cert-gdq7g
Events:
  Type    Reason     Age   From          Message
  ----    ------     ----  ----          -------
  Normal  Issuing    11m   cert-manager  Issuing certificate as Secret does not exist
  Normal  Generated  11m   cert-manager  Stored new private key in temporary Secret resource "ingress-cert-gdq7g"
  Normal  Requested  11m   cert-manager  Created new CertificateRequest resource "ingress-cert-dp6sp"

I checked the certificate request and it contains no events. Also i can see no challenges. I have added the logs below. Any help would be appreciated


kubectl describe certificaterequest ingress-cert-dp6sp -n abc

Namespace:    abc
Labels:       <none>
Annotations:  cert-manager.io/certificate-name: ingress-cert
              cert-manager.io/certificate-revision: 1
              cert-manager.io/private-key-secret-name: ingress-cert-gdq7g
API Version:  cert-manager.io/v1beta1
Kind:         CertificateRequest
Metadata:
  Creation Timestamp:  2021-09-08T07:48:33Z
  Generate Name:       ingress-cert-
  Generation:          1
  Owner References:
    API Version:           cert-manager.io/v1alpha2
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Certificate
    Name:                  ingress-cert
    UID:                   90905ab7-22d2-458c-b956-7100c4c77a8d
  Resource Version:        146643832
  Self Link:               /apis/cert-manager.io/v1beta1/namespaces/abc/certificaterequests/ingress-cert-dp6sp
  UID:                     fef72617-fc1d-4384-9f4b-a7e4502582d8
Spec:
  Issuer Ref:
    Group:  cert-manager.io
    Kind:   ClusterIssuer
    Name:   letsencrypt
  Request:  LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2Z6Q0NBV2NDQVFBd0FEQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUxMNgphTGhZNjhuNnhmMUprYlF5ek9OV1J4dGtLOXJrbjh5WUtMd2l4ZEFMVUl0TERra0t6Uksyb3loZzRMMThSQmQvCkNJaGJ5RXBYNnlRditKclRTOC84T1A0MWdwTUxBLzROdVhXWWtyeWhtZFdNaFlqa21OOFpiTUk1SlZZcVV2cVkKRWQ1b2cydmVmSjU1QlJPRExsd0o3YjBZa3hXckUwMGJxQ1ExWER6ZzFhM08yQ2JWd1NQT29WV2x6Uy9CdzRYVgpMeVdMS3E4QU52b2dZMUxXRU8xcG9YelRObm9LK2U2YVZueDJvQ1ZLdGxPaG1iYXRHYXNSaTJKL1FKK0dOWHovCnFzNXVBSlhzYVErUzlxOHIvbmVMOXNPYnN2OWd1QmxCK09yQVg2eHhkNHZUdUIwVENFU00zWis2c2MwMFNYRXAKNk01RlY3dkFFeDQyTWpuejVoa0NBd0VBQWFBNk1EZ0dDU3FHU0liM0RRRUpEakVyTUNrd0p3WURWUjBSQkNBdwpIb0ljY25kemMyZHdMbU5zYjNWa1oyRjBaUzV0YVdOeWIyWnBiaTVrWlRBTkJna3Foa2lHOXcwQkFRc0ZBQU9DCkFRRUFTQ0cwTXVHMjZRbVFlTlBFdmphNHZqUUZOVFVINWVuMkxDcXloY2ZuWmxocWpMbnJqZURuL2JTV1hwdVIKTnhXTnkxS0EwSzhtMG0rekNPbWluZlJRS1k2eHkvZU1WYkw4dTgrTGxscDEvRHl3UGxvREE2TkpVOTFPaDM3TgpDQ0E4NWphLy9FYVVvK0p5aHBzaTZuS1d4UXRpYXdmYXhuNUN4SENPWGF5Qzg0Q0IzdGZ2WWp6YUF3Ykx4akxYCmxvd09LUHNxSE51ZktFM0NtcjZmWGgramd5VWhxamYwOUJHeGxCWEFsSVNBNkN5dzZ2UmpWamFBOW82TmhaTXUKbmdheWZON00zUzBBYnAzVFFCZW8xYzc3QlFGaGZlSUE5Sk51SWtFd3EvNXppYVY1RDErNUxSSnR5ZkVpdnJLTwpmVjQ5WkpCL1BGOTdiejhJNnYvVW9CSkc2Zz09Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=
Status:
  Conditions:
    Last Transition Time:  2021-09-08T07:48:33Z
    Message:               Waiting on certificate issuance from order abc/ingress-cert-dp6sp-3843501305: ""
    Reason:                Pending
    Status:                False
    Type:                  Ready
Events:                    <none>

Here is the ingress.yaml

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 20m
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt"
spec:
  rules:
    - host: abc.xyz.com
      http:
        paths:
          - path: /static
            backend:
              serviceName: app-service
              servicePort: 80
          - path: /
            backend:
              serviceName: app-service
              servicePort: 8000
  tls:
  - hosts:
    - abc.xyz.com
    secretName: ingress-cert

Here is the clusterissuer:

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-key
    solvers:
    - http01:
        ingress:
          class: nginx

Solution

  • Works only with Nginx Ingress Controller

    I was using ClusterIssuer but I changed it to Issuer and it works.

    -- Install cert-manager (Installed version 1.6.1) and be sure that the three pods are running

    -- Create an Issuer by appling this yml be sure that the issuer is running.

    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      name: letsencrypt-nginx
      namespace: default
    spec:
     acme:
        server: https://acme-v02.api.letsencrypt.org/directory
        email: [email protected]
        privateKeySecretRef:
          name: letsencrypt-nginx-private-key
        solvers:
        - http01:
           ingress:
             class: nginx
    

    -- Add this to your ingress annotations

    cert-manager.io/issuer: letsencrypt-nginx
    

    -- Add the secretName to your ingress spec.tls.hosts spec:

      tls:
      - hosts:
        - yourdomain.com
        secretName: letsencrypt-nginx
    

    Notice that the Nginx Ingress Controller is able to generate the Certificate CRD automatically via a special annotation: cert-manager.io/issuer. This saves work and time, because you don't have to create and maintain a separate manifest for certificates as well (only the Issuer manifest is required). For other ingresses you may need to provide the Certificate CRD as well.