Search code examples
amazon-web-servicesterraformamazon-eksaws-auto-scaling

Multiple instance types per AZ configuration for ASG created by AWS EKS


I have deployed AWS EKS cluster with Spot instances using Terraform. I am using EKS managed Node Groups, as for spot instances, I want to specify multiple instance types per AZ for HA. As I can see, ASG created by EKS for us has it's launch template created, which we should not edit(as they mentioned in aws doc). I wanted to ask, is there any way I can update this "multiple instances per AZ" in my cluster. Or do I need to configure custom ASG for it? Should I rather create ASG per AZ with multiple instance types for better availability of space capacity?

I am new to EKS Spot, please let me know if any more input is needed.

Please refer CAS configuration :

resource "helm_release" "cluster-autoscaler" {
  name        = local.app
  namespace   = var.namespace
  repository  = local.cluster_autoscaler_helm_repository
  chart       = var.cluster_autoscaler_helm_chart
  version     = var.cluster_autoscaler_helm_version

  values = [
    yamlencode({
      autoDiscovery = {
        clusterName = var.cluster_name
      }
      awsRegion = var.region
      extraArgs = {
        scan-interval = var.scan_interval
        expander = "least-waste"
        skip-nodes-with-local-storage = false
        skip-nodes-with-system-pods = false
      }
      extraVolumes = [
        {
          name             = "ssl-certs"
          hostPath = {
            path = "/etc/ssl/certs/ca-bundle.crt"
          }
        }
      ]
      extraVolumeMounts = [
        {
          name          = "ssl-certs"
          readOnly     = true
          mountPath    = "/etc/ssl/certs/ca-certificates.crt"

        }
      ]
      image = {
        repository = local.cluster_autoscaler_image
        tag        = var.cluster_autoscaler_image_version
      }
      podAnnotations = {
         "cluster-autoscaler.kubernetes.io/safe-to-evict" = "false"
      }
      podLabels = {
        app = local.app
      }
      rbac = {
        serviceAccount = {
          annotations = {
            "eks.amazonaws.com/role-arn": "arn:aws:iam::${var.account_number}:role/cluster-autoscaler"
          }
        }
      }
      replicaCount = var.replica_count

      # resources -- Pod resource requests and limits.
      resources = {
        limits = {
          cpu = var.resources_limit_cpu
          memory = var.resources_limit_memory
        }
        requests = {
          cpu = var.requests_limit_cpu
          memory = var.requests_limit_memory
        }
      }

    })]

}


Solution

  • For configuring spot allocation startegy, I found in this document that, for EKS Managed Node Groups, if I am using capacity_type = "SPOT" then by default spotAllocationStrategy="capacity-optimized" is assigned. So basically, my purpose is served here.