I recently learned about withCredentials DSL thanks to an answer to this question.
Having attempted to use @RamKamath's answer, i.e. the following Jenkinsfile:
pipeline {
agent any
stages {
stage( "1" ) {
steps {
script {
def credId = "cred_id_stored_in_jenkins"
withCredentials([usernamePassword(credentialsId: credId,
passwordVariable: 'password',
usernameVariable: 'username')]) {
String url = "https://bitbucket.company.com/rest/build-status/1.0/commits"
String commit = '0000000000000000000000000000000000000001'
Map dict = [:]
dict.state = "INPROGRESS"
dict.key = "foo_002"
dict.url = "http://server:8080/blue/organizations/jenkins/job/detail/job/002/pipeline"
List command = []
command.add("curl -f -L")
command.add('-u ${username}:${password}')
command.add("-H \\\"Content-Type: application/json\\\"")
command.add("-X POST ${url}/${commit}")
command.add("-d \\\''${JsonOutput.toJson(dict)}'\\\'")
sh(script: command.join(' '))
}
}
}
}
}
}
...the curl command itself fails because of a reported "Bad request" error. This is the snippet from the Jenkins console output:
+ curl -f -L -u ****:**** -H "Content-Type:application/json" -X POST https://bitbucket.company.com/rest/build-status/1.0/commits/0000000000000000000000000000000000000001 -d '{"state":"INPROGRESS","key":"foo_002","url":"http://server:8080/blue/organizations/jenkins/job/detail/job/002/pipeline"}'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 153 0 0 100 153 0 4983 --:--:-- --:--:-- --:--:-- 5100
curl: (22) The requested URL returned error: 400 Bad request
I understand that -u ****:**** is the masked username:password argument to -u.
If I copy/paste that exact string into a shell, and replace the masked values with the real values, the curl command works:
$ curl -f -L -u super_user:super_password -H "Content-Type:application/json" -X POST https://bitbucket.company.com/rest/build-status/1.0/commits/0000000000000000000000000000000000000001 -d '{"state":"INPROGRESS","key":"foo_002","url":"http://server:8080/blue/organizations/jenkins/job/detail/job/002/pipeline"}'
$
What is going wrong? Why does the curl command result in error 400/"Bad request" when Jenkins executes it, but the same command runs fine when executed manually?
Please note: as was recommended, I enclosed the -u ${username}:${password} in single-quotes, not double-quotes.
Update: I feel as though something is wrong with the string interpolation, because if I modify the Jenkinsfile to add a hardcoded username/password, i.e.
command.add('-u super_user:super_password')
...instead of
command.add('-u ${username}:${password}')
...then the curl command still fails exactly as before, i.e. because of the error: 400 Bad request
Can someone please help me identify what is wrong, presumably with the command assembly, and/or the sh() invocation?
Update
I've simplified the problem by removing the withCredentials(). Even this simplified curl invocation fails:
pipeline {
agent any
stages {
stage( "1" ) {
steps {
script {
def credId = "cred_id_stored_in_jenkins"
String url = "https://bitbucket.company.com/rest/build-status/1.0/commits"
String commit = '0000000000000000000000000000000000000001'
Map dict = [:]
dict.state = "INPROGRESS"
dict.key = "foo_002"
dict.url = "http://server:8080/blue/organizations/jenkins/job/detail/job/002/pipeline"
List command = []
command.add("curl -f -L")
command.add('-u super_user:super_password')
command.add("-H \\\"Content-Type: application/json\\\"")
command.add("-X POST ${url}/${commit}")
command.add("-d \\\''${JsonOutput.toJson(dict)}'\\\'")
sh(script: command.join(' '))
}
}
}
}
}
This issue turned out to be a string-escaping problem. The working solution -- including withCredentials(), which was not a factor in the problem -- for me was:
pipeline {
agent any
stages {
stage( "1" ) {
steps {
script {
def credId = "cred_id_stored_in_jenkins"
String url = "https://bitbucket.company.com/rest/build-status/1.0/commits"
String commit = '0000000000000000000000000000000000000001'
withCredentials([usernamePassword(credentialsId: credId,
passwordVariable: 'password',
usernameVariable: 'username')]) {
Map dict = [:]
dict.state = "INPROGRESS"
dict.key = "foo_002"
dict.url = http://server:8080/blue/organizations/jenkins/job/detail/job/002/pipeline"
def cmd = "curl -f -L" +
"-u ${username}:${password} " +
"-H \"Content-Type: application/json\" " +
"-X POST ${url}/${commit} "
"-d \'${JsonOutput.toJson(dict)}\'")
sh(script: cmd)
}
}
}
}
}
}
I'm sure some variation of the List.join() would have worked - and there's no specific reason I reverted to using + to join the strings other than I was hacking away, and settled on the first thing that just worked. Escaping strings in Jenkins appears to be its own little circle of Hell, so I don't want to spend more time there than I need to.
A few oddities revealed themselves while working on this:
First, behavior appears to be different in Windows vs. Unix/bash: @GeroldBroser (whose help was invaluable) was able to get a working solution in his Windows environment with string-escaping closer/identical to my original post; however I was not able to reproduce his result in my Unix/bash environment (Jenkins sh invocations use bash in my setup).
Lastly, I was under the impression that the text logged to a Jenkins job console output was literally what was executed -- but this doesn't appear to be quite true.
To summarize a portion of my comment-discussion with @GeroldBroser:
The curl command, when run by Jenkins failed with error: 400 Bad request, yet if I copy/pasted/executed the exact curl command logged in my Jenkins job console ouput in a bash shell, it worked successfully.
By making use of the --trace-ascii /dev/stdout option to curl, I was able to discover that the curl command, when run successfully in bash, sent 141 bytes, but when run unsuccessfully by Jenkins, sent 143 bytes: the extra 2 bytes were leading and trailing ' (single-quote) characters before and after the JSON content.
This led me down the path of madness to the circle of Hell to the castle of damnation to the throne of insanity that is Jenkins string escaping, and I eventually arrived at the above working solution.
Noteworthy: with this working solution, I can no longer copy/paste the curl command -- as logged in my Jenkins job console output -- to a bash shell and successfully execute. Therefore, it's not (always) true that "what is logged in the Jenkins job console output is exactly what is run (i.e. copy/pastable) in the shell."