jenkins operator v0.4.0 deployed on kubernetes but we can't persiste our jobs and pipelines

Question

we face an issue with jenkins installed on kubernetes using jenkins operator, so we can't persist created jobs because after restarting the pods we lost our jobs, those are our configurations used to start it:

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: jenkins
  namespace: integration
spec:
  configurationAsCode:
    configurations:
  groovyScripts:
    configurations:
  backup:
    containerName: backup
    action:
      exec:
        command:
        - /home/user/bin/backup.sh
    interval: 30
    makeBackupBeforePodDeletion: true
  restore:
    containerName: backup
    action:
      exec:
        command:
        - /home/user/bin/restore.sh
  master:
    basePlugins:
    - name: kubernetes
      version: 1.25.2
    - name: workflow-job
      version: "2.39"
    - name: workflow-aggregator
      version: "2.6"
    - name: git
      version: 4.2.2
    - name: job-dsl
      version: "1.77"
    - name: configuration-as-code
      version: "1.38"
    - name: kubernetes-credentials-provider
      version: "0.13"
    plugins:
    - name: maven-plugin
      version: "3.8"
    - name: ansible
      version: "1.1"
    - name: bitbucket
      version: 1.1.27
    - name: bitbucket-build-status-notifier
      version: 1.4.2
    - name: docker-plugin
      version: 1.2.1
    - name: generic-webhook-trigger
      version: "1.72"
    - name: github-pullrequest
      version: 0.2.8
    - name: job-import-plugin
      version: "3.4"
    - name: msbuild
      version: "1.29"
    - name: nexus-artifact-uploader
      version: "2.13"
    - name: pipeline-npm
      version: 0.9.2
    - name: pipeline-utility-steps
      version: 2.6.1
    - name: pollscm
      version: 1.3.1
    - name: postbuild-task
      version: "1.9"
    - name: ranorex-integration
      version: 1.0.2
    - name: sidebar-link
      version: 1.11.0
    - name: sonarqube-generic-coverage
      version: "1.0"
    - name: sonar
      version: "2.13"
    - name: simple-theme-plugin
      version: "0.6"
    priorityClassName:
    disableCSRFProtection: false
    containers:
      - name:  jenkins-master
        image: jenkins/jenkins:lts
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 12
          httpGet:
            path: /login
            port: http
            scheme: HTTP
          initialDelaySeconds: 80
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 5
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /login
            port: http
            scheme: HTTP
          initialDelaySeconds: 30
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          limits:
            cpu: 1500m
            memory: 3Gi
          requests:
            cpu: 1
            memory: 500Mi
      - name: backup
        image: virtuslab/jenkins-operator-backup-pvc:v0.0.8
        imagePullPolicy: IfNotPresent
        env:
        - name: BACKUP_DIR
          value: /backup
        - name: JENKINS_HOME
          value: /jenkins-home
        - name: BACKUP_COUNT
          value: "3"
        volumeMounts:
        - mountPath: /jenkins-home
          name: jenkins-home
        - mountPath: /backup
          name: backup
    volumes:
    - name: backup
      persistentVolumeClaim:
        claimName: jenkins-backup
    - name: jenkins-home
      persistentVolumeClaim:
        claimName: jenkins-home
    securityContext:
      fsGroup: 1000
      runAsUser: 1000
  seedJobs:
    - description: Jenkins Operator repository
      id: jenkins-operator
      repositoryBranch: master
      repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git
      targets: cicd/jobs/*.jenkins

the operator have two scripts backup and restore and what we seen is that our pre-configured jobs are persisted but the new created one (using GUI) they aren't. any ideas about this problem? or the jenkins operator doesn't permit the persistency ?

Answer 1

From the docs (https://jenkinsci.github.io/kubernetes-operator/docs/getting-started/latest/configuring-backup-and-restore/):

Because of Jenkins Operator’s architecture, the configuration of Jenkins should be done using ConfigurationAsCode or GroovyScripts and jobs should be defined as SeedJobs. It means that there is no point in backing up any job configuration up. Therefore, the backup script makes a copy of jobs history only.

So yes, this is the intended behaviour, you should create new jobs as SeedJobs, not in the GUI.