Search code examples
amazon-web-servicesamazon-ecshealth-checkcloudwatch-alarms

Do CloudWatch Alarms for UnHealthyHostCount respect HealthCheckGracePeriodSeconds?


I have an alarm for unhealthy hosts which is alerting during deploys and turning the ECS service on and off. I have tried adding HealthCheckGracePeriodSeconds : 600 as suggested here which will allow for a 10 minute startup time which is more than enough time for the service to start responding to health checks, but still an alert on the alarm.

I don't really want to change the alarm because I want it to be very sensitive to any problems which might occur in the environment, but may be force to as at this point the alarms which I have set up can be a bit noisy.


Solution

  • The health check grace period is simply to ignore the result of the alarm. It means for 10 minutes it will not replace even if the alarm is going off.

    If you're alarming on UnHealthyHostCount you will need to tweak some part of your monitoring solution to avoid the situation when this crossover occurs.

    Remember that one of these is related to the ECS Service which is just listening whether it is healthy, whereas the other is entirely the load balancer having an alarm for unhealthy hosts.