We're in the process of firing up a new Jenkins instance on K8s and decided to use Helm to ease the way. Now, we're having issues with JCASC and Helm not behaving as it should as explained here.
According to the document linked above, you should be able to include multiple files from the command line which then get converted and placed into the location for the CASC files, /var/jenkins_home/casc_congigs Here is what the command should look like:
helm install jenkins -n sandbox-dev -f jenkins-values.yaml -f values_welcome_message.yaml $chart
I read and tried answers from Provide external yaml file to JCasC configScripts but so far, nothing works.
I created a system message YAML file:
jenkins:
controller:
JCasC:
configScripts:
welcome-message: |
jenkins:
systemMessage: Welcome to our CI\CD server. This Jenkins is configured and managed 'as code'.
The main values file is configured like this for JCASC:
JCasC:
enabled: true
defaultConfig: true
configUrls: []
# - https://acme.org/jenkins.yaml
# Remote URL:s for configuration files.
configScripts: {}
# welcome-message: |
# jenkins:
# systemMessage: Welcome to our CI\CD server. This Jenkins is configured and managed 'as code'.
Note that the inline configuration script for a welcome message is commented out.
I perform the installation like this:
helm install jenkins -n sandbox-dev -f jenkins-values.yaml -f values_welcome_message.yaml $chart
The additional YAML file is not uploaded to the /var/jenkins_home/casc_configs directory.
If I change the initial values file to uncomment the config for the welcome message and perform an install, all works as it should and the additional config file is created (without trying to upload the additional YAML file).
jenkins@jenkins-0:/$ ls -la /var/jenkins_home/casc_configs/
total 12
drwxrwsrwx. 2 root jenkins 67 Nov 27 20:49 .
drwxrwsrwx. 14 root jenkins 4096 Nov 27 20:49 ..
-rw-r--r--. 1 jenkins jenkins 2446 Nov 27 20:33 jcasc-default-config.yaml
-rw-r--r--. 1 jenkins jenkins 105 Nov 27 20:49 welcome-message.yaml
What am I missing here? Why is the additional file in the install command I shared not uploading the values_welcome_message.yaml? Do I have the initial values YAML set up incorrectly? Or is there something else at play?
It turns out the instructions given here are not as clear as they should be. The external files are uploaded and merged, they are just not merged into the corrected place.
The bottom line is that you must take into account the structure of the values.yaml file when you are including external files for JCASC. In the instructions shown, you might create an external file like this, for example:
jenkins:
controller:
JCasC:
configScripts:
jenkinsCasc: |
jenkins:
disableRememberMe: false
mode: NORMAL
...
Just as I had done with the welcome message YAML:
jenkins:
controller:
JCasC:
configScripts:
welcome-message: |
jenkins:
systemMessage: Welcome to our CI\CD server. This Jenkins is configured and managed 'as code'.
It turns out that the structure as shown, is wrong.
To solve this I did a dry run of the Helm install with debug output sent to a file:
helm upgrade --dry-run --debug jenkins -n sandbox-dev -f jenkins-values.yaml -f values-welcome-message.yaml $chart >> dry-run.txt
Examining the output, I found that the second file had been included for merging, but the information was not in the correct place under the JCASC: property:
controller:
JCasC:
authorizationStrategy: |-
loggedInUsersCanDoAnything:
allowAnonymousRead: false
configScripts: {}
configUrls: []
defaultConfig: true
enabled: true
security:
apiToken:
creationOfLegacyTokenEnabled: false
tokenGenerationOnCreationEnabled: false
usageStatisticsEnabled: true
securityRealm: |-
local:
allowsSignup: false
enableCaptcha: false
users:
- id: "${chart-admin-username}"
name: "Jenkins Admin"
password: "${chart-admin-password}"
But it does appear in the output with its base level being Jenkins:
jenkins:
controller:
JCasC:
configScripts:
welcome-message: |-
jenkins:
systemMessage: Welcome to our CI\CD server. This Jenkins is configured and managed 'as code'.
If you look at the original values.yaml, you will find the base level of the document is where the controller is located. Collapsing that shows the JCASC properties are a child of the controller: not of jenkins: controller:
controller:
##### REMOVED COLLAPSED LINES #####
JCasC:
enabled: true
defaultConfig: true
configUrls: []
# - https://acme.org/jenkins.yaml
# Remote URL:s for configuration files.
##### THIS SECTION IS HERE BECAUSE IT IS PART OF THE DEFAULT values.yaml FILE #####
configScripts: {}
# welcome-message: |
# jenkins:
# systemMessage: Welcome to the Newfold CI\CD server. This Jenkins is configured and managed 'as code'.
# Allows adding to the top-level security JCasC section. For legacy, default the chart includes apiToken configurations
security:
apiToken:
creationOfLegacyTokenEnabled: false
tokenGenerationOnCreationEnabled: false
usageStatisticsEnabled: true
# Ignored if securityRealm is defined in controller.JCasC.configScripts
In order to make the external JCASC files proper, they must follow the same pattern. Removing the Jenkins: parent does the trick (note the relation of the controller to the JCASC property:
controller:
JCasC:
configScripts:
welcome-message: |
jenkins:
systemMessage: Welcome to the Jenkins CI\CD server. This is managed with JCASC.
The output after another dry run shows the properties if the external file to be in the correct hierarchy to work:
controller:
JCasC:
authorizationStrategy: |-
loggedInUsersCanDoAnything:
allowAnonymousRead: false
configScripts:
welcome-message: |-
jenkins:
systemMessage: Welcome to our CI\CD server. This Jenkins is configured and managed 'as code'.
configUrls: []
defaultConfig: true
enabled: true
security:
apiToken:
creationOfLegacyTokenEnabled: false
tokenGenerationOnCreationEnabled: false
usageStatisticsEnabled: true
securityRealm: |-
local:
allowsSignup: false
enableCaptcha: false
users:
- id: "${chart-admin-username}"
name: "Jenkins Admin"
password: "${chart-admin-password}"