TL;DR
From a pod I am able to resolve DNS addresses only when I am using CoreDNS endpoint IP. How to make pods use coreDNS IP address?
CoreDNS configuration
Kubelet configuration
...
clusterDNS:
- 10.96.0.10
...
Flannel
Default configuration from: https://github.com/flannel-io/flannel
Cluster
Is up and running and have these pods:
Also, the cluster consists of only a single master node (that is schedulable).
Pod setup
# run pod
kubectl run -it --rm --restart=Never --image=infoblox/dnstools:latest dnstools
Test cases
# 1
nslookup kubernetes.default
# output: ;; connection timed out; no servers could be reached
# 2
nslookup kubernetes.default 10.96.0.10
# output: ;; connection timed out; no servers could be reached
# 3
nslookup kubernetes.default 10.244.0.222
# output: resolved address, Address: 10.96.0.1
# 4
ping 10.96.0.1
# output: all packages lost
# 5
ping 10.96.0.10
# output: all packages lost
# 6
ping 10.244.0.222
# output: all packages received
How have that happened?
That issue appeared when I was updating a cluster. I have reinstalled kube-flannel and coreDNS services with their default equivalents:
Throughout this process I have also uninstalled kube-proxy.
I don't know if kube-proxy is a necessary component of a k8s or does a flannel can be an alternative but I tried to reinstall it with this command: https://stackoverflow.com/a/71519601/19570235 However, without any luck.
# Error message:
could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt":
I was able to resolve this issue by changing clusterDNS section in kubelet from 10.96.0.10 to 10.244.0.222.
After that I have restarted kubelet service (k8s is working on Ubuntu) and pods. Now pods can resolve hostnames and other services, like google.com.
I am not really sure if that is a workaround or if it is necessary to provide coreDNS endpoint to the kubelet config and because of that I will keep this post open for a while before accepting this answer.