Route 53 health checks: CloudWatch didn't have enough data to determine the state of the alarm

Question

I have configured a health check in Route53 that monitors an alarm for API gateway. For some reason this health check doesn't change its status from healthy to unhealthy, even though the alarm is IN ALARM state. I set the alarm status using the AWS command: aws cloudwatch set-alarm-status.

Here's the config for health check:

    "HealthCheck": {
        "Id": "bxxx",
        "CallerReference": "xxxxx-xxx",
        "HealthCheckConfig": {
            "Type": "CLOUDWATCH_METRIC",
            "Inverted": false,
            "Disabled": false,
            "AlarmIdentifier": {
                "Region": "us-east-1",
                "Name": "APIGatewayAlarm"
            },
            "InsufficientDataHealthStatus": "Healthy"
        },
        "HealthCheckVersion": 9,
        "CloudWatchAlarmConfiguration": {
            "EvaluationPeriods": 1,
            "Threshold": 5.0,
            "ComparisonOperator": "GreaterThanOrEqualToThreshold",
            "Period": 60,
            "MetricName": "5XXError",
            "Namespace": "AWS/ApiGateway",
            "Statistic": "Sum",
            "Dimensions": [
                {
                    "Name": "ApiName",
                    "Value": "An API GW"
                },
                {
                    "Name": "Stage",
                    "Value": "v1"
                }
            ]
        }
    }
}

Instead, what I see in the console under Health Checkers:

Success: CloudWatch didn't have enough data to determine the state of the alarm. The Route 53 health checker considers the health check to be healthy based on the InsufficientDataHealthState setting.

Any ideas of why is there insufficient data, if the Alarm is actually in the alarm state.

Answer 1

Expected. Route 53 monitors the data stream for the corresponding alarm instead of monitoring the alarm state.

https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-determining-health-of-endpoints.html#dns-failover-determining-health-of-endpoints-cloudwatch

It does not mean you cant get check status from there.

When the alarm is in the INSUFFICIENT state, the status is unhealthy When the alarm is in the OK state, the status is healthy When the alarm is in the ALARM state, the status is unhealthy

In order not to have check reported as Unhealthy all the time do following. Set "Invert health check status" to Yes