Kubernetes's http liveness probe failed when pod under heavy load

Question

Just as the title says, Kubernetes's liveness probe always fails when pod is under heavy load(500+ concurrency, 100000 request), i gave more cpu & memory resources to the pod, but it does not work.

Is there any method to handle this situation ?

Thanks!

Answer 1

As per @Williams Mao issue was resolved by increasing readinessProbe.timeoutSeconds.

If an application has huge load, the default timeoutSeconds: 1 may be not enough. Good to read: configure-liveness-readiness-probes