Search code examples
amazon-web-serviceskuberneteskubernetes-ingressamazon-eks

How to keep LoadBalancer[ALB] even after we delete Ingress Manifest in AWS EKS?


When we launch the EKS Cluster using the below manifest, it is creating ALB. We have a default ALB that we are using, let's call it EKS-ALB. The Hosted zone is routing traffic to this EKS-ALB. We gave tag ingress.k8s.aws/resource:LoadBalancer, ingress.k8s.aws/stack:test-alb, elbv2.k8s.aws/cluster: EKS. But when we delete the manifest, it is deleting the default ALB and we need to reconfigure hosted zone again with New ALB which will get created in next deployment. Is there any way to block Ingress-controller not deleting ALB, but only deleting the listeners and Target Group?

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-nginx-rule
  namespace: test
  annotations:
    alb.ingress.kubernetes.io/group.name: test-alb
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/target-type: instance
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
    alb.ingress.kubernetes.io/ssl-redirect: '443'
    alb.ingress.kubernetes.io/healthcheck-port: traffic-port
    alb.ingress.kubernetes.io/healthcheck-path: /index.html
    alb.ingress.kubernetes.io/success-codes: 200-399
    alb.ingress.kubernetes.io/security-groups: eks-test-alb-sg
spec:
  ingressClassName: alb
  rules:
  - host: test.eks.abc.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: test-svc
            port:
              number: 5005
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-dep
  namespace: test
  labels:
    app: test
spec:
  replicas: 1
  restartPolicy:
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
      containers:
      - name: test
        image: Imagepath
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 5005
        resources:
          requests:
            memory: "256Mi"
            cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
  name: test-svc
  namespace: test
  labels:
    app: test
spec:
  type: NodePort
  ports:
  - port: 5005
    targetPort: 80
    protocol: TCP
  selector:
    app: test
---
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: test-scaler
  namespace: test
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: test-dep
  minReplicas: 1
  maxReplicas: 5
  targetCPUUtilizationPercentage: 60
---

Solution

  • In order to achieve the existing ALB not being deleted with group.name annotation enabled, we need to meet following conditions:

    1. ALB should be tagged with below 3 tags:
    alb.ingress.kubernetes.io/group.name: test-alb
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/target-type: instance
    
    1. Create a dummy ingress with the same group name with the below manifest.
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: test-nginx-rule
      namespace: test
      annotations:
        alb.ingress.kubernetes.io/group.name: test-alb
        alb.ingress.kubernetes.io/scheme: internal
        alb.ingress.kubernetes.io/target-type: instance
        alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
        alb.ingress.kubernetes.io/ssl-redirect: '443'
        alb.ingress.kubernetes.io/healthcheck-port: traffic-port
        alb.ingress.kubernetes.io/healthcheck-path: /index.html
        alb.ingress.kubernetes.io/success-codes: 200-399
        alb.ingress.kubernetes.io/security-groups: eks-test-alb-sg
    spec:
      ingressClassName: alb
      rules:
      - host: dummy.eks.abc.com
        http:
          paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: test-svc
                port:
                  number: 5005
    

    After deploying the above manifest, an ingress will be created using the same ALB and listener will have rule of if host is dummy.eks.abc.com, it will return 443. It's create and forget type of manifest, so after creating this ingress, even after we delete all the running deployment services (except the dummy manifest file above), the ALB will remain.