Search code examples
amazon-web-servicesamazon-eksaws-application-load-balancer

EKS ALB is not to able to auto-discover subnets


Background:

  • I have a VPC with 3 public subnets(the subnets have access to an internet gateway)

  • I have an EKS Cluster in this VPC, the EKS cluster is created from the console and not using eksctl

  • I used this tutorial from the official aws documentation, I managed to set my ALB controller and the controller is running perfectly:

The cluster contains two node groups:

  • First node group has one node of type: t3a.micro
  • Second node group has one node of type: t3.small
$ kubectl get deployment -n kube-system aws-load-balancer-controller
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
aws-load-balancer-controller   1/1     1            1           60m

I used their game example and here is the manifest file:

---
apiVersion: v1
kind: Namespace
metadata:
  name: game-2048
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: game-2048
  name: deployment-2048
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: app-2048
  replicas: 1
  template:
    metadata:
      labels:
        app.kubernetes.io/name: app-2048
    spec:
      containers:
      - image: alexwhen/docker-2048
        imagePullPolicy: Always
        name: app-2048
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  namespace: game-2048
  name: service-2048
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
  type: NodePort
  selector:
    app.kubernetes.io/name: app-2048
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  namespace: game-2048
  name: ingress-2048
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  rules:
    - http:
        paths:
          - path: /*
            backend:
              serviceName: service-2048
              servicePort: 80

However when I describe ingress: I get the following messages

DNDT@DNDT-DEV-2 MINGW64 ~/Desktop/.k8s
$ kubectl describe ingress/ingress-2048 -n game-2048
Name:             ingress-2048
Namespace:        game-2048
Address:
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *
              /*   service-2048:80 (172.31.4.64:80)
Annotations:  alb.ingress.kubernetes.io/scheme: internet-facing
              alb.ingress.kubernetes.io/target-type: ip
              kubernetes.io/ingress.class: alb
Events:
  Type     Reason            Age                From     Message
  ----     ------            ----               ----     -------
  Warning  FailedBuildModel  9s (x13 over 32s)  ingress  Failed build model due to couldn't auto-discover subnets: unable to discover at least one subnet

Here are the tags set on the 3 subnets: enter image description here

And here are the route table for the subnets, as you can see they have an internet gw attached: enter image description here

I searched everywhere and they all talk about adding the tags, I created a completely new cluster from scratch but still getting this issue, are there any other things I'm missing?

I checked this answer, but its not relevant because its for ELB not ALB,

================================

Update:

I explicitly added the subnets:

alb.ingress.kubernetes.io/subnets: subnet-xxxxxx, subnet-xxxxx, subnet-xxx

And now I got my external IP, but with some warning

$  kubectl describe ingress/ingress-2048 -n game-2048
Name:             ingress-2048
Namespace:        game-2048
Address:          k8s-game2048-ingress2-330cc1efad-115981283.eu-central-1.elb.amazonaws.com
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *
              /*   service-2048:80 (172.31.13.183:80)
Annotations:  alb.ingress.kubernetes.io/scheme: internet-facing
              alb.ingress.kubernetes.io/subnets: subnet-8ea768e4, subnet-bf2821f2, subnet-7c023801
              alb.ingress.kubernetes.io/target-type: ip
              kubernetes.io/ingress.class: alb
Events:
  Type     Reason             Age   From     Message
  ----     ------             ----  ----     -------
  Warning  FailedDeployModel  43s   ingress  Failed deploy model due to ListenerNotFound: One or more listeners not found
           status code: 400, request id: e866eba4-328c-4282-a399-4e68f55ee266
  Normal   SuccessfullyReconciled  43s  ingress  Successfully reconciled

Also going to the browser and using the external ip return: 503 Service Temporarily Unavailable


Solution

  • Ensure that --cluster-name in the aws-load-balancer-controller deployment is correct configured.

    Use

    kubectl get deployment -n kube-system aws-load-balancer-controller -oyaml |grep "cluster-name"

    to get the cluster name in the deployment.

    If it isn't correct, edit deployment with next command and rename it:

    kubectl edit deployment -n kube-system aws-load-balancer-controller