Traefik 2.3 with custom SSL certificates won't work

Question

I've setup a Kubernetes cluster using GKE with an instance of Traefik as the ingress-controller. So far everything is ok, but my SSL certificates are not found by Traefik.

My CustomResourceDefinition file:

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressrouteudps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteUDP
    plural: ingressrouteudps
    singular: ingressrouteudp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsstores.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSStore
    plural: tlsstores
    singular: tlsstore
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: traefikservices.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TraefikService
    plural: traefikservices
    singular: traefikservice
  scope: Namespaced

My RBAC file:

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - middlewares
      - ingressroutes
      - traefikservices
      - ingressroutetcps
      - ingressrouteudps
      - tlsoptions
      - tlsstores
    verbs:
      - get
      - list
      - watch

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: default

My Service Account:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller

My ConfigMap:

kind: ConfigMap
apiVersion: v1
metadata:
  name: traefik-config-map
data:
  traefik.toml: |-
    [global]
      checkNewVersion = false
      sendAnonymousUsage = false
    [retry]
      attempts = 3
      maxMem = 3
    [entryPoints]
      [entryPoints.web]
        address = ":80"
        [entryPoints.web.http]
          [entryPoints.web.http.redirections]
            [entryPoints.web.http.redirections.entryPoint]
              to = "websecure"
              scheme = "https"
              permanent = true
      [entryPoints.websecure]
        address = ":443"
    [log]
      level = "DEBUG"
    [accessLog]
    [api]
      insecure = true
      dashboard = true
      debug = true
    [providers]
      [providers.file]
        directory = "/etc/traefik"
        watch = true
      [providers.kubernetesCRD]


    [[tls.certificates]]
       keyFile = "/var/ssl/tls.key"
       certFile = "/var/ssl/tls.crt"

    [tls.options]
        [tls.options.default]
          minVersion = "VersionTLS12"
          preferServerCipherSuites = true
          cipherSuites = [
            "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
            "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
            "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305",
            "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
            "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
            "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA",
            "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
            "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
            "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
          ]

        [tls.options.mintls13]
          minVersion = "VersionTLS13"

    [tls.stores]
      [tls.stores.default]
        [tls.stores.default.defaultCertificate]
          keyFile = "/var/ssl/tls.key"
          certFile = "/var/ssl/tls.crt"

My Deployment:

---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik
  labels:
    app: traefik
spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      volumes:
        - name: config
          configMap:
            name: traefik-config-map
        - name: samisaude-tls
          secret:
            secretName: samisaude-tls
      containers:
        - name: traefik
          image: traefik:v2.3.7
          ports:
            - name: web
              containerPort: 80
            - name: admin
              containerPort: 8080
            - name: websecure
              containerPort: 443
          volumeMounts:
            - mountPath: /etc/traefik/traefik.toml
              name: config
              subPath: traefik.toml
            - mountPath: "/var/ssl"
              name: samisaude-tls
              readOnly: true

My Secret:

apiVersion: v1
data:
  tls.key: <ENCODED_KEY_HERE>
  tls.crt: <ENCODED_CRT_HERE>
kind: Secret
metadata:
  name: samisaude-tls
  namespace: default
type: Opaque

My Service:

apiVersion: v1
kind: Service
metadata:
  name: traefik
spec:
  type: LoadBalancer
  selector:
    app: traefik
  ports:
    - protocol: TCP
      port: 80
      name: web
      targetPort: 80
    - protocol: TCP
      port: 443
      name: websecure
      targetPort: 80
    - protocol: TCP
      port: 8080
      name: admin
      targetPort: 8080

For testing purposes, I uploaded a whoami container.

Deployment:

kind: Deployment
apiVersion: apps/v1
metadata:
  namespace: default
  name: whoami
  labels:
    app: whoami
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
        - name: whoami
          image: containous/whoami
          ports:
            - name: web
              containerPort: 80

Ingress:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: whoami
  namespace: default
spec:
  tls: {}
  entryPoints:
    - web
    - websecure
  routes:
  - match: Host(`mysubdomain.samisaude.com.br`) && PathPrefix(`/whoami`)
    kind: Rule
    services:
    - name: whoami
      port: 80

Service:

apiVersion: v1
kind: Service
metadata:
  name: whoami
spec:
  ports:
    - protocol: TCP
      name: web
      port: 80
  selector:
    app: whoami

In Traefik's Dashboard everything is ok, but when accessing my URL in the browser the certificate is still Traefik's default and not what I added. In the Traefik POD logs, I get the following message:

time="2021-09-28T16:33:31Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2021-09-28T16:33:31Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd

Any help is welcome. Thanks.

Answer 1

The following would work. I had tried with GCP. You can also follow the link

00-resource-crd-definition.yml

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressrouteudps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteUDP
    plural: ingressrouteudps
    singular: ingressrouteudp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsstores.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSStore
    plural: tlsstores
    singular: tlsstore
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: traefikservices.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TraefikService
    plural: traefikservices
    singular: traefikservice
  scope: Namespaced

05-traefik-rbac.yml

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - middlewares
      - ingressroutes
      - traefikservices
      - ingressroutetcps
      - ingressrouteudps
      - tlsoptions
      - tlsstores
    verbs:
      - get
      - list
      - watch

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: default
---

10-service-account.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller

11-traefik-configmap.yaml

kind: ConfigMap
apiVersion: v1
metadata:
  name: traefik-config-map
data:
  traefik.toml: |-
    [global]
      checkNewVersion = false
      sendAnonymousUsage = false
    [retry]
      attempts = 3
      maxMem = 3
    [entryPoints]
      [entryPoints.web]
        address = ":80"
        [entryPoints.web.http]
          [entryPoints.web.http.redirections]
            [entryPoints.web.http.redirections.entryPoint]
              to = "websecure"
              scheme = "https"
              permanent = true
      [entryPoints.websecure]
        address = ":443"
    [log]
      level = "DEBUG"
    [accessLog]
    [api]
      insecure = true
      dashboard = true
      debug = true
    [providers]
      [providers.file]
        directory = "/etc/traefik"
        watch = true
      [providers.kubernetesCRD]
    [[tls.certificates]]
       keyFile = "/var/ssl/start-domain-com-ssl/star_domain.com.key"
       certFile = "/var/ssl/start-domain-com-ssl/star_domain_com.chained.crt"
    [tls.options]
        [tls.options.default]
          minVersion = "VersionTLS12"
          preferServerCipherSuites = true
          cipherSuites = [
            "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
            "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
            "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305",
            "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
            "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
            "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA",
            "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
            "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
            "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
          ]
        [tls.options.mintls13]
          minVersion = "VersionTLS13"
    [tls.stores]
      [tls.stores.default]
        [tls.stores.default.defaultCertificate]
          keyFile = "/var/ssl/start-domain-com-ssl/star_domain.com.key"
          certFile = "/var/ssl/start-domain-com-ssl/star_domain_com.chained.crt"

12-secret.yaml

apiVersion: v1
data:
  star_domain.com.key: <Add base64 encoded value of you cert.key>
  star_domain_com.chained.crt: <Add base64 encoded value of your chained cert.crt>
kind: Secret
metadata:
  name: start-domain-com-ssl
  namespace: default
type: Opaque

15-traefik-deployment.yaml

---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik
  labels:
    app: traefik

spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      volumes:
        - name: config
          configMap:
            name: traefik-config-map
        - name: start-domain-com-ssl
          secret:
            secretName: start-domain-com-ssl
      containers:
        - name: traefik
          image: traefik:v2.2.1
          ports:
            - name: web
              containerPort: 80
            - name: admin
              containerPort: 8080
            - name: websecure
              containerPort: 443
          volumeMounts:
            - mountPath: /etc/traefik/traefik.toml
              name: config
              subPath: traefik.toml
            - mountPath: "/var/ssl/start-domain-com-ssl"
              name: start-domain-com-ssl
              readOnly: true

20-traefik-service.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: traefik
spec:
  type: LoadBalancer
  selector:
    app: traefik
  ports:
    - protocol: TCP
      port: 80
      name: web
      targetPort: 80
    - protocol: TCP
      port: 443
      name: websecure
      targetPort: 80
    - protocol: TCP
      port: 8080
      name: admin
      targetPort: 8080

25-whoami-deployment.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  namespace: default
  name: whoami
  labels:
    app: whoami

spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
        - name: whoami
          image: containous/whoami
          ports:
            - name: web
              containerPort: 80

30-whoami-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: whoami

spec:
  ports:
    - protocol: TCP
      name: web
      port: 80
  selector:
    app: whoami

35-whoami-ingress-route.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: whoami-whoami
  namespace: default
spec:
  tls: {}
  entryPoints:
    - web
    - websecure
  routes:
  - match: PathPrefix(`/whoami-app-api`)
    kind: Rule
    services:
    - name: whoami
      port: 80