Search code examples
kubernetesistiocert-managermetallb

Istio routing, metallb and https issue

I am having a problem with kubernetes K3S, Istio, MetalLB and CertManager.

I have my cluster hosted on a VPS with one public ip. As my service provider don provide me with a Load Balancer, i am using MetlLb with my public Ip to reach internet with istio-ingressgateway.

In thsis cluster i have three namespaces for my applications, one for qa environment, othe for dev and the prod environment.

I configured my ip in my dns provider with my public ip, and configured CertManager to get a Certificate from letsencrypt (i am using Issuer instead of ClusterIssuer as i want to use the staging api for dev and qa and prod for prod). Certificate are issued fine, but the Gateway from istio is routing the traffic only when i use the port 80, when i enable the 443 i cant reach the site by https, getting a "ERR_CONNECTION_RESET".

I cant understand why is everyhing fine for 80, but not for the 443. My application es exposing the traffic in the port 80 by http.

Here are my yaml files:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-v1
spec:
  replicas: 3
  selector:
    matchLabels:
      app: hello-v1
  template:
    metadata:
      labels:
        app: hello-v1
    spec:
      containers:
      - name: hello
        image: pablin.dynu.net:5000/chevaca/chevacaweb:latest
        imagePullPolicy: IfNotPresent        
        ports:
        - containerPort: 80
        resources:
          requests:
            memory: "64Mi"
            cpu: "200m"
          limits:
            memory: "128Mi"
            cpu: "500m"

kind: Service
apiVersion: v1
metadata:
  name: hello-v1-svc
  namespace: chevaca-qa
spec:
  selector:
    app: hello-v1
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: qa-app-gateway
  namespace: chevaca-qa
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        name: http
        number: 80
        protocol: HTTP
      hosts:
        - qa-app.chevaca.com
    - port:
        name: https
        number: 443
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: front-cert
      hosts:
        - qa-app.chevaca.com

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: front-app
  namespace: chevaca-qa
spec:
  hosts:
    - qa-app.chevaca.com
  gateways:
    - qa-app-gateway
  http:
    - route:
      - destination:
          host: hello-v1-svc
          port:
            number: 80
Solution

  • Its fixed,

    the solution is to create the certificate in the istio-system namespace with the secret name ingressgateway-certs.

    In that way, the certificate is mounted in the ingress gateway service from istio, and nothing else need to be configured on the Custom Resources from istio. If you have multiple namespaces , like my scenario, you can use multiple hosts on the certificate or you can use a wild card.