Search code examples
amazon-web-servicesamazon-eksnlbaws-nlb

EKS - Network Load Balancer exposes internal IP - how to expose it to the Internet?


I created a LoadBalancerservice in EKS, but it exposes internal IP addresses. I am not interested to use ALB, I want to use NLB and terminate connections in Pods.

Pods are running on Fargate nodes.

Could anyone of you suggest what else needs to be done to expose this service to the public Internet, please?

Service definition:

apiVersion: v1
kind: Service
metadata:
  name: echo-server
  namespace: dev-echo-server
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb-ip"
spec:
  selector:
    app: echo-server
  ports:
    - protocol: TCP
      port: 8080
      targetPort: http-8080
  type: LoadBalancer
  sessionAffinity: None
  externalTrafficPolicy: Cluster
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack
  allocateLoadBalancerNodePorts: false
  loadBalancerClass: service.k8s.aws/nlb
  internalTrafficPolicy: Cluster

External IP addresses belong to internal class, and they are not visible from the Internet.

$ kubectl get svc
NAME          TYPE           CLUSTER-IP      EXTERNAL-IP                                                                     PORT(S)    AGE
echo-server   LoadBalancer   172.20.246.19   k8s-devechos-echoserv-930c12fb8d-cd88a6c85735d503.elb.eu-west-1.amazonaws.com   8080/TCP   6m36s

$ host k8s-devechos-echoserv-930c12fb8d-cd88a6c85735d503.elb.eu-west-1.amazonaws.com
k8s-devechos-echoserv-930c12fb8d-cd88a6c85735d503.elb.eu-west-1.amazonaws.com has address 10.0.16.198

Additional information Terraform definition to create VPC:

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 4.0"

  name = local.name
  cidr = local.vpc_cidr

  azs             = local.azs
  private_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 4, k)]
  public_subnets  = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 48)]
  intra_subnets   = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 52)]

  enable_nat_gateway = true
  single_nat_gateway = true

  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }

  tags = local.tags
}

Here is the Load Balancer description:

{
    "LoadBalancers": [
        {
            "LoadBalancerArn": "arn:aws:elasticloadbalancing:eu-west-1:493334834594:loadbalancer/net/k8s-devechos-echoserv-930c12fb8d/cd88a6c85735d503",
            "DNSName": "k8s-devechos-echoserv-930c12fb8d-cd88a6c85735d503.elb.eu-west-1.amazonaws.com",
            "CanonicalHostedZoneId": "Z2IFOLAFXWLO4F",
            "CreatedTime": "2023-08-14T12:34:50.645000+00:00",
            "LoadBalancerName": "k8s-devechos-echoserv-930c12fb8d",
            "Scheme": "internal",
            "VpcId": "vpc-032abfa8b12ae2a87",
            "State": {
                "Code": "active"
            },
            "Type": "network",
            "AvailabilityZones": [
                {
                    "ZoneName": "eu-west-1a",
                    "SubnetId": "subnet-08a705830583829fc",
                    "LoadBalancerAddresses": []
                },
                {
                    "ZoneName": "eu-west-1b",
                    "SubnetId": "subnet-08e3697bb06aae5e7",
                    "LoadBalancerAddresses": []
                },
                {
                    "ZoneName": "eu-west-1c",
                    "SubnetId": "subnet-0b7c6812480cb628f",
                    "LoadBalancerAddresses": []
                }
            ],
            "IpAddressType": "ipv4"
        }
    ]
}```

Solution

  • You need to add the following annotation to your Service:

    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    

    For this to work properly, your public subnets need to tagged with the kubernetes.io/role/elb: 1 key/value pair.