Why delete is attempted on the security group provided in alb.ingress.kubernetes.io/security-groups annotation

Question

I have an EKS cluster and I am using aws-load-balancer-controller:v2.1.1. I am using the below aws alb ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-alb-ingress
  namespace: myns
  labels: 
    app: myns-alb-ingress
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/tags: Team=myteam
    alb.ingress.kubernetes.io/group.name: my-apps
    alb.ingress.kubernetes.io/certificate-arn: arn:xxx
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}, {"HTTP":80}]'
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'

Also there is a security group "sg-xxxxc91467c97bbb" with the description "[k8s] Managed SecurityGroup for LoadBalancer".

I have a bunch of IP addresses added in this Security Group as inbound rules. Now whenever I add any host rules (path/ backend service ) and apply this ingress, all the inbound rules of this security group gets deleted and 0.0.0.0/0 gets added automatically.

How can I prevent this from happening so that I retain my inbound rules?

I tried adding

alb.ingress.kubernetes.io/security-groups: sg-xxxxc91467c97bbb

to the annotations above and it retains all the rules. However in the output of

kubectl logs -n kube-system   deployment.apps/aws-load-balancer-controller

I see logs like

{"level":"info","ts":1615287229.3065767,"logger":"controllers.ingress","msg":"deleting securityGroup","securityGroupID":"sg-xxxxc91467c97bbb"}

and in the ingress describe, I see event like

Events:
  Type     Reason             Age                    From     Message
  ----     ------             ----                   ----     -------
  Warning  FailedDeployModel  5m2s (x9 over 20m)  ingress  Failed deploy model due to failed to delete securityGroup: timed out waiting for the condition

So if I provide the security group in the annotations, the inbound rules are retained but something tries to delete the security group itself which I don't want. Any suggestions?

Answer 1

I had faced similar issues so I created a new security group (sg-xxxxxxx042) with all necessary inbound rules and passed it to annotation as follows.

alb.ingress.kubernetes.io/security-groups: sg-xxxxxxx042

along with that, I needed to add one more inbound rule to security groups attached to the eks worker nodes (ec2 instances)

In my case security group name starts with eks-remoteAccess-**** which is attached to the instance.

I added the following inbound rule

Alltraffic All All custom sg-xxxxxxx042