I'm not sure how it's possible, but I set up a Route 53 health check with email alerting if our endpoint goes down.
It is definitely down because the EC2 hosting it is powered off.
❯ telnet foo.io 443
Trying 18.18.18.18...
telnet: connect to address 18.18.18.18: Operation timed out
telnet: Unable to connect to remote host
Is it possible that the checker has cached something? Although we don't use anything in between and it's supposed to hit the EC2 directly.
I think you have left your health check disabled
That's what the doc states
Stops Route 53 from performing health checks. When you disable a health check, Route 53 stops aggregating the status of the referenced health checks.
After you disable a health check, Route 53 considers the status of the health check to always be healthy. If you configured DNS failover, Route 53 continues to route traffic to the corresponding resources.
Maybe that's why you see it passing