AWS ELB is extremely slow and patchy

Question

I have setup internet-facing ELB to access webserver of Apache airflow, which runs in 8080 of the instance.

Configuration

1. Single-AZ ELB
2. Autoscaling group with single m4.large instance

Below is the terraform resource for the ELB

resource "aws_elb" "airflow_elb" {
  name = "${var.domain_name}-elb"
  subnets = [
    "${aws_subnet.private.id}"]

  security_groups = [
    "${aws_security_group.public.id}"]

  "listener" {
    instance_port = 8080
    instance_protocol = "http"
    lb_port = 80
    lb_protocol = "http"
  }

  health_check {
    healthy_threshold = "${var.elb_healthy_threshold}"
    interval = "${var.elb_interval}"
    target = "HTTP:8080/admin/"
    timeout = "${var.elb_timeout}"
    unhealthy_threshold = "${var.elb_unhealthy_threshold}"
  }

  access_logs {
    bucket = "${aws_s3_bucket.bucket.bucket}"
    bucket_prefix = "elb-logs"
    interval = 60
  }

  cross_zone_load_balancing = false
  idle_timeout = 400
  connection_draining = true
  connection_draining_timeout = 400



  tags {
    Name = "airflow-elb"
  }

}

I can ssh tunnel to the private-ip instance via bastion host and the portal works without any issue. But when I access via the DNS name of the ELB it is either extremely slow, in which case I can see the request is reponded almost instantaneously from the webserver, but takes forever to load or ELB throws HTTP 503

Please help!!

EDIT1: Backend processing time is very high, but I can see that happens only when accessed from ELB, when done from tunneled connection it behaves normally.

Answer 1

The issue was actually with using sync worker with python 3 and how ELB reuses http connection. The issue disappeared after changing from sync worker to gevent. However gevent is not supported as of yet by python 3, so we r stuck with python 2.7 for now