Cloudbees 2.289.1.2
I am changing the code as per the Jenkinsfile string interpolation guidelines. Note: I am aware that Groovy String interpolation expects double-quotes for variables to be included.
There are several sh in several global variables(under vars/) in a shared library.
Currently(without using String interpolation directives by Jenkins), the commands specified below give a warning(shown after each command).
First command: This command can't populate custom/user-defined variables but it seems that username and password are populated:
String resultingImage = "${repo}/${parameters.openshiftProject}/${appImageName}"
.toLowerCase()
// tag images
withCredentials([usernamePassword(
credentialsId: pushCredentialsId,
passwordVariable: 'password',
usernameVariable: 'username')]) {
parameters[IMAGE_TAGS].each { tagName ->
sh 'set +x skopeo copy docker://$resultingImage:latest docker://$resultingImage:$tagName --src-tls-verify=false --src-creds=$username:$password --dest-creds=$username:$password echo "done copying ${resultingImage}:${tagName}"'
}
}
Output:
[Pipeline] withCredentials
Masking supported pattern matches of $username or $password
[Pipeline] {
[Pipeline] sh
+ set +x skopeo copy docker://:latest docker://: --src-tls-verify=false '--src-creds=****:****' '--dest-creds=****:****' echo 'done copying :'
If double-quotes are used, the following warning is shown but the command is formed correctly:
[Pipeline] {
[Pipeline] sh
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
Affected argument(s) used the following variable(s): [password, username]
See https://jenkins.io/redirect/groovy-string-interpolation for details.
+ set +x
Second command: This throws an error(seems that the credentials are retrieved correctly), no matter if I use "${chartTarballName}" or ${chartTarballName} or $chartTarballName:
sh 'curl -f -u $username:$password -T "${chartTarballName}" $chartPublishRepo/$chartTarballName'
The error:
[Pipeline] sh
+ curl -f -u ****:**** -T '' /
curl: (3) <url> malformed
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
ERROR: script returned exit code 3
If double-quotes are used, the following warning is shown but the command is formed correctly:
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
Affected argument(s) used the following variable(s): [password, username]
See https://jenkins.io/redirect/groovy-string-interpolation for details.
+ curl -f -u ****:**** -T exp-calculator-0.14.11.tgz https://repository.net/artifactory/stable-local/exp-calculator-0.14.11.tgz
I guess that these interpolation rules work only within a declarative pipeline directive, and not for scripted pipelines, global variables(at least, the user-defined variables) in a shared library, and other Groovy DSL scripts.
Update-1:
The single-quote seems to work only for the standard, directive pipeline directives:
pipeline {
environment {
CUSTOM_ENV_VAR = 'This is an environment variable'
}
agent any
stages {
stage('build') {
steps {
withCredentials([string(credentialsId: 'sq_token', variable: 'SECRET')]) {
sh ("echo $SECRET")
sh ('echo $SECRET $CUSTOM_ENV_VAR')
script {
String scriptBlockVariable = 'This is a variable in a script block'
sh ('echo $scriptBlockVariable')
sh ('echo $SECRET $scriptBlockVariable')
}
}
}
}
}
}
Output:
[Pipeline] stage
[Pipeline] { (build)
[Pipeline] withCredentials
Masking supported pattern matches of $SECRET
[Pipeline] {
[Pipeline] sh
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
Affected argument(s) used the following variable(s): [SECRET]
See https://jenkins.io/redirect/groovy-string-interpolation for details.
+ echo ****
****
[Pipeline] sh
+ echo **** This is an environment variable
**** This is an environment variable
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ echo
[Pipeline] sh
+ echo ****
****
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // stage
Try to look at it as follows: when you execute a job that uses interpreters such as the sh, bat and powershell, there are two separate contexts that hold parameters for the execution.
The first context is the actual groovy code that holds all parameters that where defined by the code, it could be input parameters, global parameters or any parameter that was defined in your code using the def keyword.
The second context is the set of parameters that will be loaded into the execution runtime environment whenever you run an interpreter step (sh, bat, powershell), this parameters will be available to these steps as environment parameters.
The string interpolation of groovy runs in the first context and can access only variables defined in the code, while commands that are passed and executed by the shell steps can access only parameters from the second context. In your case you are mixing both of them together.
Now let's examine the different keywords and their scope:
A simple def statement for defining a parameter will make it available for the first context - groovy code and thus string interpolation.
parameters block in the declarative pipeline is a special keyword that makes the parameters available in both contexts - it is available to the groovy code but is also loaded as an environment variables to the runtime environment of shell steps environment.It supports both regular parameters and credentials.
Scripted pipelines can't use the parameters directive, instead they have a different keyword for adding parameters to execution environment which is called withEnv:
withEnv- Sets one or more environment variables within a block. These are available to any external processes spawned within that scope.
withEnv supports only regular parameters and therefore another keyword was introduced for supporting credentials: the withCredentials that enables to set credentials parameters alike the withEnv step.
Jenkins indeed recommends not to use string interpolation for sensitive parameters, but rather pass them to the execution environment (context 2) and lets the shell steps use them as environment variables, so for that you need to define your string with single quotes, but on the other hand you want to use parameters defined in your code (context 1) which can be evaluated only with string interpolation that requires double quotes. So you are actually trying to use parameters from both context simultaneously which is a bit tricky.
You have two options for handling this mixture:
1 - use the withCredentials alongside the withEnv keyword, define the string with single quotes (disable string interpolation) using shell syntax parameter reference ($) and let the shell extract all the parameters from the environment variables. Something like:
String resultingImage = "${repo}/${parameters.openshiftProject}/${appImageName}".toLowerCase()
// tag images
withCredentials([usernamePassword(credentialsId: pushCredentialsId, passwordVariable: 'password',usernameVariable: 'username')]) {
parameters[IMAGE_TAGS].each { tagName ->
withEnv(['TAG_NAME=tagName', "IMAGE=${resultingImage}"]) {
sh 'set +x skopeo copy docker://$IMAGE:latest docker://$IMAGE:$TAG_NAME --src-tls-verify=false --src-creds=$username:$password --dest-creds=$username:$password echo "done copying $IMAGE:$TAG_NAME"'
}
}
}
2 - separate the creation of the command string into parts, each handling the correct context - groovy parameters will be evaluated when the command is created and the only the relevant parameters will be extract by the shell command. Something like:
String resultingImage = "${repo}/${parameters.openshiftProject}/${appImageName}".toLowerCase()
// tag images
withCredentials([usernamePassword(credentialsId: pushCredentialsId, passwordVariable: 'password', usernameVariable: 'username')]) {
parameters[IMAGE_TAGS].each { tagName ->
sh 'set +x skopeo copy docker://' + resultingImage + ':latest docker://' + "${resultingImage}:${tagName}" + '--src-tls-verify=false --src-creds=$username:$password --dest-creds=$username:$password echo' + /"done copying ${resultingImage}:${tagName}"/
}
}
Here is a very nice article on Jenkins environment variables and usage.