Search code examples
dockerstatushealth-check

How can i get my container to go from starting -> healthy


Background: My Docker container has a very long startup time, and it is hard to predict when it is done. And when the health check kicks in, it first may show 'unhealthy' since the startup is sometimes not finished. This may cause a restart or container removal from our automation tools.

My specific question is if I can control my Docker container so that it shows 'starting' until the setup is ready and that the health check can somehow be started immediately after that? Or is there any other recommendation on how to handle states in a good way using health checks?

Side question: I would love to get a reference to how transitions are made and determined during container startup and health check initiating. I have tried googling how to determine Docker (container) states but I can't find any good reference.


Solution

  • My specific question is if I can control my container so that it shows 'starting' until the setup is ready and that the health check can somehow be started immediately after that?

    I don't think that it is possible with just K8s or Docker.
    Containers are not designed to communicate with Docker Daemon or Kubernetes to tell that its internal setup is done.
    If the application takes a time to setup you could play with readiness and liveness probe options of Kubernetes.

    You may indeed configure readynessProbe to perform the initial check after a specific delay.
    For example to specify 120 seconds as initial delay :

    readinessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 120
    

    Same thing for livenessProbe:

    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
        httpHeaders:
        - name: Custom-Header
          value: Awesome
      initialDelaySeconds: 120
      periodSeconds: 3
    

    For Docker "alone" while not so much configurable you could make it to work with the --health-start-period parameter of the docker run sub command :

    --health-start-period : Start period for the container to initialize before starting health-retries countdown

    For example you could specify an important value such as :

    docker run --health-start-period=120s ...