Search code examples
dockerkubernetesnetworkingkind

How to access kind control plane port from another docker container?


I'm creating a kind cluster with kind create cluster --name kind and I want to access it from another docker container but when I try to apply a Kubernetes file from a container (kubectl apply -f deployment.yml) I got this error:

The connection to the server 127.0.0.1:6445 was refused - did you specify the right host or port?

Indeed when I try to curl kind control-plane from a container, it's unreachable.

> docker run --entrypoint curl curlimages/curl:latest 127.0.0.1:6445
curl: (7) Failed to connect to 127.0.0.1 port 6445 after 0 ms: Connection refused

However kind control-plane is publishing to the right port but only to the localhost.

> docker ps --format "table {{.Image}}\t{{.Ports}}"
IMAGE                  PORTS
kindest/node:v1.23.4   127.0.0.1:6445->6443/tcp

Currently the only solution I found is to set the host network mode.

> docker run --network host --entrypoint curl curlimages/curl:latest 127.0.0.1:6445
Client sent an HTTP request to an HTTPS server.

This solution don't look to be the most secure. Is there another way like connecting the kind network to my container or something like that that I missed ?


Solution

  • Don't have enough rep to comment on the other answer, but wanted to comment on what ultimately worked for me.

    Takeaways

    • Kind cluster running in it own bridge network kind
    • Service with kubernetes client running in another container with a mounted kube config volume
    • As described above the containers need to be in the same network unless you want your service to run in the host network.
    • The server address for the kubeconfig is the container name + internal port e.g. kind-control-plane:6443. The port is NOT the exposed port in the example below 6443 NOT 38669
      CONTAINER ID   IMAGE                                PORTS
      7f2ee0c1bd9a   kindest/node:v1.25.3                 127.0.0.1:38669->6443/tcp
      

    Kube config for the container

    # path/to/some/kube/config
    apiVersion: v1
    clusters:
      - cluster:
          insecure-skip-tls-verify: true # Don't use in Prod equivalent of --insecure on cli
          server: https://<kind-control-plane container name>:6443 # NOTE port is internal container port
        name: kind-kind # or whatever
    contexts:
      - context:
          cluster: kind-kind
          user: <some-service-account>
        name: kind-kind # or whatever
    current-context: kind-kind
    kind: Config
    preferences: {}
    users:
      - name: <some-service-account>
        user:
          token: <TOKEN>
    

    Docker container stuff

    • If using docker-compose you can add the kind network to the container such as:

      #docker-compose.yml
      services:
        foobar:
          build:
            context: ./.config
          networks:
            - kind # add this container to the kind network
          volumes:
            - path/to/some/kube/config:/somewhere/in/the/container
      networks:
        kind: # define the kind network
          external: true # specifies that the network already exists in docker
      
    • If running a new container:

      docker run --network kind -v path/to/some/kube/config:/somewhere/in/the/container <image>
      
    • Container already running?

      docker network connect kind <container name>