Search code examples
kubernetesyamlcontainerskubectlkubernetes-pod

Kubernetes CrashLoopBackOff when running a sidecar container


When running a sidecar container along with the main container in my kubernets deployment I'm getting an error with the timeout when mounting the volume.

MountVolume.SetUp failed for volume "initdb" : failed to sync configmap cache: timed out waiting for the condition

kubectl describe pod mypod

.
.
.
  client:
    Container ID:   docker://xx
    Image:          docker.io/golang:xx
    Image ID:       docker-xxx
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Tue, 14 Jun 2022 15:44:22 +0200
      Finished:     Tue, 14 Jun 2022 15:44:22 +0200
    Ready:          False
    Restart Count:  3
    Environment:
      env_a:                <set to the key 'env_a' of config map 'unleash'>      Optional: false
      token:  <set to the key 'token' in secret 'mysecrets'>  Optional: false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from xxx (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  initdb:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      initdb
    Optional:  false
  kube-api-access-rrrsv:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           xxx
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason       Age                    From               Message
  ----     ------       ----                   ----               -------
  Normal   Scheduled    6m44s                  default-scheduler  Successfully xxx to minikube
  Warning  FailedMount  6m42s                  kubelet            MountVolume.SetUp failed for volume "initdb" : failed to sync configmap cache: timed out waiting for the condition
  Normal   Pulled       3m23s (x5 over 6m)     kubelet            Container image "xxx/postgres:xx" already present on machine
  Normal   Created      3m21s (x5 over 5m56s)  kubelet            Created container init-db
  Normal   Started      3m14s (x5 over 5m45s)  kubelet            Started container init-db
  Warning  BackOff      3m12s (x9 over 5m1s)   kubelet            Back-off restarting failed container
  Normal   Pulled       94s                    kubelet            Container image "xx/unleash-server:xxx" already present on machine

This is the deployment file that I'm trying to deploy:

I did not attach all the yaml file as it's working fine except when including the sidecarcontainer.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/name: mydeployment
  name: mydeployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: mydeployment
  template:
    metadata:
      labels:
        app.kubernetes.io/name: mydeployment
    spec:
      volumes:
        - name: volume-db
          configMap:
            name: volume-db
      initContainers:
        - name: datbase
          image: "myrepo/postgres:xxx"
          env:
            .
            .
            .
            # environment variables to configure the postgres database. Not attached here #
            .
            .
            .
          volumeMounts:
            - mountPath: /etc/postgresql
              name: volume-db
              readOnly: true
      containers:
        - name: maincontainer
          env:
            .
            .
            .
            # environment variables to configure the maincontainer. Not attached here #
            .
            .
            .
          image: "myrepo/maincontainer:xxx"
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          livenessProbe:
            httpGet:
              path: localhost/maincontainer/health
              port: http
            initialDelaySeconds: 20
            timeoutSeconds: 250
          readinessProbe:
            httpGet:
              path: localhost/maincontainer/health
              port: http
            initialDelaySeconds: 20
            timeoutSeconds: 250
            successThreshold: 5
            periodSeconds: 10
        - name: sidecarcontainer
          env:
            .
            .
            .
            #environment variables to configure the sidecar container. Not attached here#
            .
            .
            .
          image: "myrepo/sidecarcontainer:xxx"
          imagePullPolicy: IfNotPresent

Any idea why is throwing the timeout error when mounting the volume?

Is it something related to the timeout when including the sidecar container and needs to be incresed?

If it's something related on how I'm including the sidecar container a clear example will be appreciated.


Solution

  • I've already fixed the issue.

    Since the sidecar container is running a go image it's necessary to run some command so that the pod won´t finished its execution.

    Adding these lines at the end of the sidecar container definition in the deployment yaml file fixed the issue:

    - name: sidecarcontainer
              env:
                .
                .
                .
                #environment variables to configure the sidecar container. Not attached here#
                .
                .
                .
              image: "myrepo/sidecarcontainer:xxx"
              imagePullPolicy: IfNotPresent
              command: ["bash"]
              args: ["-c", "while true; do echo hello; sleep 300; done"]