Search code examples
kubernetescorednsmicrok8s

Kubernetes not able pull image from private registry having private domain pointed via /etc/hosts

I am trying to deploy an image from my private registry that's hosted on my local network and pointed using my local machines /etc/hosts file.

I am getting the resolution error as below:

Failed to pull image "gitlab.example.com:5050/group/project:latest": rpc error: code = Unknown desc = failed to resolve image

My /etc/hosts file contains:

192.168.1.100 gitlab.example.com

Using docker the pull/push works perfectly fine as the resolution happens using /etc/hosts

I've tried editing corefile of coredns to make the resolution happen, but it isn't working.

Can someone point me in right direction over here.

Solution

  • You can try to use hostAliases 💡 and add a host entry to your pod. Kubernetes will not honor the /etc/hosts file from the hosts/nodes. Either the one in the pod or resolves through CoreDNS. For example:

    apiVersion: v1
kind: Pod
metadata:
  name: hostaliases-pod
spec:
  restartPolicy: Never
  hostAliases:
  - ip: "192.168.1.100" 👈
    hostnames:
    - "gitlab.example.com" 👈
  containers:
  - name: gitlab-hosts
    image: myimage
    command:
    - mygitlabjob
    args:
    - "arg1"

    Update:

    I think see the problem here. microk8s on Ubuntu runs in a snap. That means it's confined/sandboxed in a container of its own. This also means that it probably doesn't care about your machine's /etc/hosts file. Unfortunately, snap's file systems are mounted as read-only for security reasons and to prevent tampering.

    ○ → pwd
/snap/microk8s/current
○ → sudo touch hosts
touch: cannot touch 'hosts': Read-only file system

    If you'd like to use a private registry this way, some recommendations:

    • Ask your system admin to add that entry into your local DNS server, or add it if you are the system admin.
    • Use an alternative small K8s distro that uses Docker.
    • Build your own microk8s snap with a modified /etc/hosts file (hard)