Search code examples
kubernetesetcd

Connection refused to etcd in Kubernetes


Question

The reason why etcd in Kubernetes refuses connections and accept only from local is by design for security?

Master node

$ sudo netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:2379          0.0.0.0:*               LISTEN      386/etcd            
tcp        0      0 127.0.0.1:2380          0.0.0.0:*               LISTEN      386/etcd            

$ hostname
master

$ ping master
PING master (192.168.99.10) 56(84) bytes of data.
64 bytes from master (192.168.99.10): icmp_seq=1 ttl=64 time=0.028 ms
64 bytes from master (192.168.99.10): icmp_seq=2 ttl=64 time=0.059 ms
^C

$ curl -iv  http://master:2379/health
curl: (7) Failed connect to master:2379; Connection refused

$ curl -s http://localhost:2379/health | python -m json.tool
{
    "health": "true"
}

Worker node

$ curl -iv http://192.168.99.10:2379/health
* Failed connect to 192.168.99.10:2379; Connection refused

Solution

  • By design only kube-api Server is able to access the ectd Server. Kube-api server can change the state of the cluster since etcd is known as the source of truth in the kubernertes cluster.

    Each request to etcd server is authenticated, authorized, validated as well as converted into etcd schema for stability and reliability as well as security purpose.

    Therefore, it's better to send the requests directly to kube-api server which will do all of the above functionality for you.

    You can communicate with etcd server by using etcdctl as well. It is used for checking cluster health as well as generating database snapshots. for example,

    etcdctl cluster-health