how does K8S handles multiple remote docker registeries in POD definition using imagePullSecrets list

Question

I would like to access multiple remote registries to pull images. In the k8s documentation they say:

(If you need access to multiple registries, you can create one secret for each registry. Kubelet will merge any imagePullSecrets into a single virtual .docker/config.json)

and so the POD definition should be something like this:

apiVersion: v1
kind: Pod
spec:
  containers:
    - name: ...
  imagePullSecrets:
    - name: secret1
    - name: secret2
    - ....
    - name: secretN

Now I am not sure how K8S will pick the right secret for each image? will all secrets be verified one by one each time? and how K8S will handle the failed retries? and if a specific amount of unauthorized retries could lead to some lock state in k8sor docker registries?

/ Thanks

Answer 1

Kubernetes isn't going to try all secrets until find the correct. When you create the secret, you are referencing that it's a docker registry:

$ kubectl create secret docker-registry user1-secret --docker-server=https://index.docker.io/v1/ --docker-username=user1 --docker-password=PASSWORD456 --docker-email=user1@email.com

$ kubectl create secret docker-registry user2-secret --docker-server=https://index.docker.io/v1/  --docker-username=user2 --docker-password=PASSWORD123 --docker-email=user2@email.com

$ kubectl get secrets user1-secret -o yaml
apiVersion: v1
data:
  .dockerconfigjson: eyJhdXRocyI6eyJkb2NrZXIuZXhhbXBsZS5jb20iOnsidXNlcm5hbWUiOiJrdWJlIiwicGFzc3dvcmQiOiJQV19TVFJJTkciLCJlbWFpbCI6Im15QGVtYWlsLmNvbSIsImF1dGgiOiJhM1ZpWlRwUVYxOVRWRkpKVGtjPSJ9fX0=
kind: Secret
metadata:
  creationTimestamp: "2020-01-13T13:15:52Z"
  name: user1-secret
  namespace: default
  resourceVersion: "1515301"
  selfLink: /api/v1/namespaces/default/secrets/user1-secret
  uid: d2f3bb0c-3606-11ea-a202-42010a8000ad
type: kubernetes.io/dockerconfigjson

As you can see, type is kubernetes.io/dockerconfigjson is telling Kubernetes to treat this differently.

So, when you reference the address of your container as magic.example.com/magic-image on your yaml, Kubernetes will have enough information to connect the dots and use the right secret to pull your image.

apiVersion: v1
kind: Pod
metadata:
  name: busyboxes
  namespace: default
spec:
  imagePullSecrets:
  - name: user1-secret
  - name: user2-secret
  containers:
  - name: jenkins
    image: user1/jenkins
    imagePullPolicy: Always
  - name: busybox
    image: user2/busybox
    imagePullPolicy: Always    

So as this example describes, it's possible to have 2 or more docker registry secrets with the same --docker-server value. Kubernetes will manage to take care of it seamlessly.