jenkinskubernetes-helmcicd

Helm JCASC Config Not Merging Additional Files


We're in the process of firing up a new Jenkins instance on K8s and decided to use Helm to ease the way. Now, we're having issues with JCASC and Helm not behaving as it should as explained here.

According to the document linked above, you should be able to include multiple files from the command line which then get converted and placed into the location for the CASC files, /var/jenkins_home/casc_congigs Here is what the command should look like:

helm install jenkins -n sandbox-dev -f jenkins-values.yaml -f values_welcome_message.yaml $chart

I read and tried answers from Provide external yaml file to JCasC configScripts but so far, nothing works.

I created a system message YAML file:

jenkins:
  controller:
    JCasC:
      configScripts:
        welcome-message: |
          jenkins:
            systemMessage: Welcome to our CI\CD server.  This Jenkins is configured and managed 'as code'.

The main values file is configured like this for JCASC:

JCasC:
    enabled: true
    defaultConfig: true
    configUrls: []
    # - https://acme.org/jenkins.yaml
    # Remote URL:s for configuration files.
    configScripts: {}
      # welcome-message: |
      #  jenkins:
      #   systemMessage: Welcome to our CI\CD server.  This Jenkins is configured and managed 'as code'.

Note that the inline configuration script for a welcome message is commented out.

I perform the installation like this:

helm install jenkins -n sandbox-dev -f jenkins-values.yaml -f values_welcome_message.yaml $chart

The additional YAML file is not uploaded to the /var/jenkins_home/casc_configs directory.

If I change the initial values file to uncomment the config for the welcome message and perform an install, all works as it should and the additional config file is created (without trying to upload the additional YAML file).

jenkins@jenkins-0:/$ ls -la /var/jenkins_home/casc_configs/
total 12
drwxrwsrwx.  2 root    jenkins   67 Nov 27 20:49 .
drwxrwsrwx. 14 root    jenkins 4096 Nov 27 20:49 ..
-rw-r--r--.  1 jenkins jenkins 2446 Nov 27 20:33 jcasc-default-config.yaml
-rw-r--r--.  1 jenkins jenkins  105 Nov 27 20:49 welcome-message.yaml

What am I missing here? Why is the additional file in the install command I shared not uploading the values_welcome_message.yaml? Do I have the initial values YAML set up incorrectly? Or is there something else at play?


Solution

  • It turns out the instructions given here are not as clear as they should be. The external files are uploaded and merged, they are just not merged into the corrected place.

    The bottom line is that you must take into account the structure of the values.yaml file when you are including external files for JCASC. In the instructions shown, you might create an external file like this, for example:

    jenkins:
      controller:
        JCasC:
          configScripts:
            jenkinsCasc:  |
              jenkins:
                disableRememberMe: false
                mode: NORMAL
                ...
    

    Just as I had done with the welcome message YAML:

    jenkins:
      controller:
        JCasC:
          configScripts:
            welcome-message: |
              jenkins:
                systemMessage: Welcome to our CI\CD server.  This Jenkins is configured and managed 'as code'.
    

    It turns out that the structure as shown, is wrong.

    To solve this I did a dry run of the Helm install with debug output sent to a file:

    helm upgrade --dry-run --debug jenkins -n sandbox-dev -f jenkins-values.yaml -f values-welcome-message.yaml $chart >> dry-run.txt
    

    Examining the output, I found that the second file had been included for merging, but the information was not in the correct place under the JCASC: property:

    controller:
      JCasC:
        authorizationStrategy: |-
          loggedInUsersCanDoAnything:
            allowAnonymousRead: false
        configScripts: {}
        configUrls: []
        defaultConfig: true
        enabled: true
        security:
          apiToken:
            creationOfLegacyTokenEnabled: false
            tokenGenerationOnCreationEnabled: false
            usageStatisticsEnabled: true
        securityRealm: |-
          local:
            allowsSignup: false
            enableCaptcha: false
            users:
            - id: "${chart-admin-username}"
              name: "Jenkins Admin"
              password: "${chart-admin-password}"
    
    

    But it does appear in the output with its base level being Jenkins:

    jenkins:
      controller:
        JCasC:
          configScripts:
            welcome-message: |-
              jenkins:
                systemMessage: Welcome to our CI\CD server.  This Jenkins is configured and managed 'as code'.
    
    

    If you look at the original values.yaml, you will find the base level of the document is where the controller is located. Collapsing that shows the JCASC properties are a child of the controller: not of jenkins: controller:

    controller:
    ##### REMOVED COLLAPSED LINES #####  
      JCasC:
        enabled: true
        defaultConfig: true
        configUrls: []
        # - https://acme.org/jenkins.yaml
        # Remote URL:s for configuration files.
    ##### THIS SECTION IS HERE BECAUSE IT IS PART OF THE DEFAULT values.yaml FILE #####
        configScripts: {}
          # welcome-message: |
          #   jenkins:
          #     systemMessage: Welcome to the Newfold CI\CD server.  This Jenkins is configured and managed 'as code'.
        # Allows adding to the top-level security JCasC section. For legacy,  default the chart includes apiToken configurations
        security:
          apiToken:
            creationOfLegacyTokenEnabled: false
            tokenGenerationOnCreationEnabled: false
            usageStatisticsEnabled: true
        # Ignored if securityRealm is defined in controller.JCasC.configScripts
    

    In order to make the external JCASC files proper, they must follow the same pattern. Removing the Jenkins: parent does the trick (note the relation of the controller to the JCASC property:

    controller:
      JCasC:
        configScripts:
          welcome-message: |
            jenkins:
              systemMessage: Welcome to the Jenkins CI\CD server. This is managed with JCASC.
    

    The output after another dry run shows the properties if the external file to be in the correct hierarchy to work:

    controller:
      JCasC:
        authorizationStrategy: |-
          loggedInUsersCanDoAnything:
            allowAnonymousRead: false
        configScripts:
          welcome-message: |-
            jenkins:
              systemMessage: Welcome to our CI\CD server.  This Jenkins is configured and managed 'as code'.
        configUrls: []
        defaultConfig: true
        enabled: true
        security:
          apiToken:
            creationOfLegacyTokenEnabled: false
            tokenGenerationOnCreationEnabled: false
            usageStatisticsEnabled: true
        securityRealm: |-
          local:
            allowsSignup: false
            enableCaptcha: false
            users:
            - id: "${chart-admin-username}"
              name: "Jenkins Admin"
              password: "${chart-admin-password}"