Search code examples
testingkuberneteskubernetes-helm

Helm deployment status if tests failed


Is it possible to make Helm charts deployment to fail if test which is run before installation fails? Because now despite the test fails, status is 'Deployed'.

My test, which checks if MongoDB is deployed and is reachable:

apiVersion: v1
kind: Pod
metadata:
  name: "{{ .Release.Name }}-database-connection-test"
  annotations:
    "helm.sh/hook": pre-install,test-success
    "helm.sh/hook-delete-policy": before-hook-creation
spec:
  containers:
  - name: {{ .Release.Name }}-database-connection-test
    image: {{ template "mongo.image" . }}
    imagePullPolicy: Always
    env:
      - name: HOST
        value: {{ .Values.mongo.host }}
      - name: PORT
        value: {{ .Values.mongo.port | quote }}
      - name: DATABASE_NAME
        value: {{ .Values.mongo.databasename }}
      - name: USERNAME
        value: {{ .Values.mongo.username }}
      - name: PASSWORD
        value: {{ .Values.mongo.password }}
    command: ["sh", "-c", "mongo --username $USERNAME --password $PASSWORD --authenticationDatabase $DATABASE_NAME --host $HOST --port $PORT"]
  restartPolicy: Never

Solution

  • So in general this can be achieved setting resource type as Job. Job will be blocking Tiller until it will complete.
    There is a small issue here: if job will not complete it will be blocking Helm chart deployment infinite amount of time. To avoid that, need to set spec.activeDeadlineSeconds. It will timeout the job if it will not complete until the set time limit.

    apiVersion: batch/v1
    kind: Job
    metadata:
      name: "{{ .Release.Name }}-database-connection-test"
      annotations:
        "helm.sh/hook": pre-install,test-success
        "helm.sh/hook-delete-policy": before-hook-creation
    spec:
      ttlSecondsAfterFinished: 300
      backoffPolicy: 1
      activeDeadlineSeconds: 100
      template:
        spec:
          containers:
          - name: {{ .Release.Name }}-database-connection-test
            image: {{ template "mongo.image" . }}
            imagePullPolicy: Always
            env:
              - name: HOST
                value: {{ .Values.mongo.host }}
              - name: PORT
                value: {{ .Values.mongo.port | quote }}
              - name: DATABASE_NAME
                value: {{ .Values.mongo.databasename }}
              - name: USERNAME
                value: {{ .Values.mongo.username }}
              - name: PASSWORD
                value: {{ .Values.mongo.password }}
            command: ["sh", "-c", "mongo --username $USERNAME --password $PASSWORD --authenticationDatabase $DATABASE_NAME --host $HOST --port $PORT"]
          restartPolicy: Never
    

    It's kind of a workaround, because initially Helm test annotation shouldn't be used alongside other hooks.