Search code examples
network-programmingkubernetesdnsopenshiftkube-dns

Openshift/Kubernates kube dns best practise (ndots = 5)

I have been using Openshift/Kubernates for some time and this has been the understanding. For service to service communication

  • use DNS name of ${service-name} if they are under the same namespace
  • use DNS name of ${service-name}.${namespace}.svc.cluster.local if they are from different namespaces (network is joined)

Recently i was introduced with the topic of "we should add a dot after the svc.cluster.local to make it FQDN, for better DNS lookup speed". Done some testing and indeed with lookup is much faster with the dot. (~100ms without dot, 10ms with dot)

After some research, it was caused by the default dns setting from the kubernates

sh-4.2$ cat /etc/resolv.conf
search ${namespace}.svc.cluster.local svc.cluster.local cluster.local
nameserver X.X.X.X
options ndots:5

the ndots = 5 will perform a local search (sequential) if the dns name does not contain 5 dots. In the case of ${service-name}.${namespace}.svc.cluster.local, the local search will be as such

  1. ${service-name}.${namespace}.svc.cluster.local + ${namespace}.svc.cluster.local // FAILED LOOKUP
  2. ${service-name}.${namespace}.svc.cluster.local + svc.cluster.local // FAILED LOOKUP
  3. ${service-name}.${namespace}.svc.cluster.local + cluster.local // FAILED LOOKUP
  4. ${service-name}.${namespace}.svc.cluster.local // SUCCESS LOOKUP

And for ${service-name}.${namespace}.svc.cluster.local., the local search will be as such

  1. ${service-name}.${namespace}.svc.cluster.local // SUCCESS LOOKUP

References

  1. link
  2. how to debug

Questions:

  1. Since the ndots = 5 is the default setting for kubernetes, why ${service-name}.${namespace}.svc.cluster.local. is not documented on the official side ?
  2. Should we change all service call to ${service-name}.${namespace}.svc.cluster.local. ? any potential downsides ?
Solution

  • Based on the latest document here, it states that that we should use ${service}.${namespace} to call a service from different namespace and expect to resolve on the second attempt