Search code examples
node.jsherokucloudmicroserviceshealth-check

Heroku - restart on failed health check


Heroku does not support health checks on its own. It will restart services that crashed, but there is nothing like health checks.

It sometimes happen that service become unresponsive, but the process is still running. In most of modern cloud solution, you can provide health endpoint which is periodically called by the cloud hosting service and if that endpoints return either error or not at all, it will shut down such service and start new one.

That seems like industrial standard these days, but I am unable to find any solution to this for Heroku. I can even use external service with Heroku CLI, but just calling some endpoint is not sufficient - if there are multiple instances, they all share same URL and load balancer calls one of them randomly -> therefore it is possible to not hit failed instance at all. Even when I hit it, usually the health checks have something like "after 3 failed health checks in a row restart that instance", which is highly unprobable if there are 10 instances and one of it become unhealthy.

Do you have any solution to this?


Solution

  • You are right that this is industry standard and shame that it's not provided out of box.

    I can think of 2 solutions (both involve running some extra code that does all of this:

    a) use heroku API which allows you to get the IP of individual dynos, and then you can call each dyno how you want

    b) in each dyno instance you can send a request to webserver like https://iamaalive.com/?dyno=${process.env.HEROKU_DYNO_ID}