Search code examples
pythondockerkubernetes

Python Kubernetes API, fails to list nodes with Connection refused


I would like to start using the Kubernetes python API, so I decided to start by running a simple program to list all of the nodes running (only one should be). However, it does not work. Here's what I did:

I have a python file

from kubernetes import config, client

config.load_incluster_config()
v1 = client.CoreV1Api()
ret = v1.list_node()
for i in ret.items:
    print(i.metadata.name)

a Dockerfile

FROM python

RUN pip install kubernetes

COPY main.py /
COPY requirements.txt /

CMD [ "python", "main.py" ]

and a Kubernetes yaml file

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: python-docker
    imagePullPolicy: IfNotPresent

I then setup a cluster kind create cluster --name first-kind build the python docker iamge docker build -t python-docker -f Dockerfile . load the image kind load docker-image python-docker --name first-kind and apply the Kubernetes yaml kubectl apply -f pod.yaml. When I view the output kubectl logs my-pod I see the following error

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 196, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 495, in _make_request
    conn.request(
  File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 398, in request
    self.endheaders()
  File "/usr/local/lib/python3.12/http/client.py", line 1331, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.12/http/client.py", line 1091, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.12/http/client.py", line 1035, in send
    self.connect()
  File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 236, in connect
    self.sock = self._new_conn()
                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 211, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7b6436ca2de0>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "//main.py", line 15, in <module>
    ret = v1.list_node()
          ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/kubernetes/client/api/core_v1_api.py", line 17005, in list_node
    return self.list_node_with_http_info(**kwargs)  # noqa: E501
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/kubernetes/client/api/core_v1_api.py", line 17116, in list_node_with_http_info
    return self.api_client.call_api(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/kubernetes/client/api_client.py", line 348, in call_api
    return self.__call_api(resource_path, method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/kubernetes/client/api_client.py", line 180, in __call_api
    response_data = self.request(
                    ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/kubernetes/client/api_client.py", line 373, in request
    return self.rest_client.GET(url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/kubernetes/client/rest.py", line 244, in GET
    return self.request("GET", url,
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/kubernetes/client/rest.py", line 217, in request
    r = self.pool_manager.request(method, url,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/_request_methods.py", line 136, in request
    return self.request_encode_url(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/_request_methods.py", line 183, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/poolmanager.py", line 443, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 873, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 873, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 873, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: /api/v1/nodes (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7b6436ca2de0>: Failed to establish a new connection: [Errno 111] Connection refused'))

Any ideas what I did wrong?


Solution

  • You need to give the Pod a service account that has permissions to list nodes from the Kubernetes API.

    I updated your pod.yaml to include a service account, clusterrole and clusterrole binding and then tested with kind and it works fine.

    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: foo-cluster-role
    rules:
    - apiGroups: [""]
      resources: ["nodes"]
      verbs: ["get", "watch", "list"]
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: foo-binding
    subjects:
    - kind: ServiceAccount
      name: foo
      namespace: default
    roleRef:
      kind: ClusterRole
      name: foo-cluster-role
      apiGroup: rbac.authorization.k8s.io
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: foo
    ---
    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod
    spec:
      serviceAccount: foo
      containers:
      - name: my-container
        image: python-docker
        imagePullPolicy: IfNotPresent
    
    $ kubectl apply -f pod.yaml 
    clusterrole.rbac.authorization.k8s.io/foo-cluster-role created
    clusterrolebinding.rbac.authorization.k8s.io/foo-binding created
    serviceaccount/foo created
    pod/my-pod created
    
    $ kubectl logs -f my-pod   
    first-kind-control-plane