Search code examples
kubernetesopensuse

How to find reason why k8s node in Ready,SchedulingDisabled

Every day something changes my node changes its status to Ready,SchedulingDisabled. I checked kubectl describe node mynode and found nothing. kubectl uncordon helps for few hours only. How I can troubleshoot my cluster? I using opensuse kubic, cluster created with kubicctl and salt Thank you.

Solution

  • As per the Api docs the boolean flag get updated to true via a patch request to the Node API /api/v1/nodes/nodename exposed by Kubernetes API Server. There will also be an event in kubectl events when that happens

    + kubectl get events
LAST SEEN   TYPE     REASON               OBJECT                    MESSAGE
5m26s       Normal   NodeNotSchedulable   node/kind-control-plane   Node kind-control-plane status is now: NodeNotSchedulable

    So increase the logging verbosity of Kubernetes API Server to monitor and find out when that patch request is coming to the API and correlate if any activity(cordon) is going on in the cluster during that time.