How Jenkins is passing username and password credentials for checkout pull operations

Question

I have Jenkins / Mercurial latest versions. Machine is Red Hat Linux 6.6.

I'm using Release plugin in Jenkins. Maven plugins (maven-scm-plugin, maven-version-plugin and maven-enforcer-plugin) for doing release process on a project. All of these versions are using latest available versions and configuration is setup correctly.

In Jenkins job, I'm checking out the source code from a Project which sits behind RhodeCode (Mercurial hg).

Output shows like:

06:00:02 Started by timer
06:00:02 [EnvInject] - Loading node environment variables.
06:00:03 Building on master in workspace /main/jenkins/instance2/workspace/MyCoolProject
06:00:06 [MyCoolProject] $ hg --config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https" showconfig paths.default
06:00:06 [MyCoolProject] $ hg --config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https" pull --rev default
06:00:12 pulling from http://mercurialserver.my.company.com:9001/csa/MyCoolProject/
06:00:12 no changes found
06:00:12 [MyCoolProject] $ hg --config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https" update --clean --rev default
06:00:13 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
06:00:13 [MyCoolProject] $ hg --config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https" --config extensions.purge= clean --all
06:00:13 [MyCoolProject] $ hg --config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https" log --rev . --template {node}
06:00:13 [MyCoolProject] $ hg --config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https" log --rev . --template {rev}
06:00:13 [MyCoolProject] $ hg --config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https" log --rev 

As you see above, because we specified a user to perform the read/checkout operation in Mercurial DVCS tool, Jenkins passed bunch of parameters while initiating the hg commands i.e.

--config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https"

I'm trying to find out how I can send the same, when I'm using Maven SCM plugin and it's goals.

scm:checkin
scm:tag
etc which when called, initiates the underlaying version control commands (in my case, hg branch, hg outgoing, hg push commands).

Right now, the above hg commands are failing due to the fact that either the user doesn't have ~/.hgrc file containing username and password variable set OR the user does NOT have "WRITE" access to the target RhodeCode/Mercurial repository.

20:16:26 [INFO] --- maven-scm-plugin:1.9.4:checkin (default-cli) @ MyCoolProject ---
20:16:27 [INFO] EXECUTING: /bin/sh -c cd /main/jenkins/instance2/workspace/MyCoolProject && hg branch
20:16:27 [INFO] EXECUTING: /bin/sh -c cd /main/jenkins/instance2/workspace/MyCoolProject && hg outgoing
20:16:29 [INFO] EXECUTING: /bin/sh -c cd /main/jenkins/instance2/workspace/MyCoolProject && hg status
20:16:29 [INFO] [pom.xml:modified]
20:16:29 [INFO] [jenkins-MyCoolProject-43.appVersion.txt:unknown]
20:16:29 [INFO] [pom.xml.versionsBackup:unknown]
20:16:29 [INFO] EXECUTING: /bin/sh -c cd /main/jenkins/instance2/workspace/MyCoolProject && hg commit --message '"CM Jenkins - Release plugin auto check-in and creation of release tag = 0.0.29'
20:16:30 [INFO] EXECUTING: /bin/sh -c cd /main/jenkins/instance2/workspace/MyCoolProject && hg push http://cmprod2merc.my.company.com:9001/csa/MyCoolProject
20:16:34 [ERROR] 
20:16:34 EXECUTION FAILED
20:16:34   Execution of cmd : push failed with exit code: 255.
20:16:34   Working directory was: 
20:16:34     /main/jenkins/instance2/workspace/MyCoolProject
20:16:34   Your Hg installation seems to be valid and complete.
20:16:34     Hg version: 1.9.2 (OK)

• I tried configuring ~/.hgrc file (as per Mercurial docs) and everything works if the username you specify in this file has valid WRITE access on the target source code repository/project then the above errors message won't come.

• This can also be resolved if I install/configure keyring and mercurial_keyring extension/plugin on the build machine (which is easy as per the online mercurial keyring documentation).

What I'm trying to find out is:
How can I pass the --config xxxxx parameters (like Jenkins is passing while doing checkout / pull operations - coming due to the setting values we have entered under Source Code Management in Jenkins job) to the hg commands which are called when Maven SCM plugin is initiating scm:checkin / scm:tag goals which ends up calling hg commands (hg outgoing, hg push) / all hg commands without setting ~/.hgrc and installing mercurial_keyring?

Answer 1

Found the solution:

1. Even when the user (jenkins) had valid WRITE access on the target RhodeCode/Merurial repository and either ~/.hgrc or mercurial_keyring setup wsa configured successfully (i.e. username/password less hg operations after one time manual entry), Maven SCM plugin scm:checkin and scm:tag operation were still failing.

The reason for that is scm:checkin / scm:tag goals (of Maven SCM plugin) calls the version tool commands (hg commands in my case) but it was NOT passing the authentication params (username/password). For that either I could have added the and values in project pom.xml OR in ~/.m2/settings.xml withing maven-scm-plugin plugin's configuration ---OR (more secured way is to) create/use 2 new Jenkins global level variables (of password type) to create username/password variables and use / pass them to scm:checkin / scm:tag goals while calling these goals within Release plugin's configuration settings in Jenkins i.e. -Dusername=$username and -Dpassword=$password (as the values are coming from Jenkins, they will be masked automatically).

I went with the Jenkins route and create 2 password type global variables in Jenkins Global configuration under "Configure system" > Global parameters/passwords section and just passed them to scm:checkin/tag goals while calling them in Invoke Maven step (within Release plugin configuration in Jenkins).

I found, if you have ~/.hgrc set with just username in it, then Jenkins checkout /pull commands started to fail as Jenkins process stopped using the credentials what I was using to pull/clone the source code (it seems like it was giving preference to the ~/.hgrc username first as Jenkins job/process was running with jenkins user and because it didn't had a password variable/field set in ~/.hgrc, the pull/clone failed for some reason (it should have given preference to the user/credentials what we specify in the job's configuration itself). If I moved ~/.hgrc to ~/.hgrc-backup, then checkout/pull/clone worked in Jenkins worked fine (as it used the credentials what I mentioned in the Source Code Management section for Mercurial) BUT, it still failed during Maven SCM plugin not handling the under laying hg commands.

PS: running "hg push" on the workspace was working successfully (standalone, at command line) but when Maven SCM plugin was calling these goals and the goals were calling the hg commands, it didn't work for some reason.

Solution was:

1. pass -Dusername=$username -Dpassword=$password variable to scm:checkin/scm:tag goals
   

2. Make sure ~/.hgrc had username / password variables set --OR mercurial_keyring set to work with the repository (without prompting for username and password).
   

3. The reason, Maven SCM plugin goals .. which called hg commands didn't work was due to an issue with the plugin I guess. A work around is to call these goals with -DpushChanges=false and this way, the goals will not call the under laying version control push operation and thus it'll succeed. Then you have to manually add another step in the "Release Plugin in Jenkins's configuration" as "Execute Shell / Execute Windows Batch command" way to run "hg push". Then, it will work and in this case, you don't need to pass -Dusername and -Dpassword parameters to these goals.