I'm running Jenkins 2.155 on an Ubuntu 16.04, using docker.
I have around 20 Jenkins JobDSL seed jobs which have been running without issue for a couple of years now.
As of about three days ago, these are failing. I've eventually narrowed the issue down to lacking the Slack baseUrl parameter.
Initially the failure output was different, but then we tried updating Jenkins as well as all plugins, as this was due anyway, and the error changed. But in any case, we didn't make any changes prior to the initial error.
This is an example of the code in the DSL script, though there any many similar that have started failing.
/**
* Job Config
* Each job should provide
* id - unique id of the job
* serverName - The Jenkins name of the server which needs to be connected snapshotted
**/
def webservers = [
[id: 'server1-automated-snapshot', serverName: 'Server 1'],
[id: 'server2-automated-snapshot', serverName: 'Server 2'],
[id: 'server3', serverName: 'Server 3'],
[id: 'server 4', serverName: 'Server 4'],
]
/**
* Job Template
* The job template for automated snapshot jobs. Changing this will update all the snapshot jobs.
*/
for(webserver in webservers) {
def jobName = webserver.id
jobDisplayName = webserver.serverName+' automated volume snapshot'
def jobDescription = """This job automatically runs the automated snapshot script located on the $webserver.serverName once a day."""
job(jobName) {
description(jobDescription)
displayName(jobDisplayName)
logRotator {
numToKeep(5)
}
configure {
it / 'properties' << 'hudson.plugins.disk__usage.DiskUsageProperty' {}
}
wrappers {
timestamps()
}
triggers {
cron('H 2 * * 1')
}
steps {
publishOverSsh {
server(webserver.serverName) {
transferSet {
execCommand('sudo /usr/local/sbin/snapshot.sh')
}
}
}
}
publishers {
mailer('techadmin@ibboost.com', true, false)
retryBuild {
rerunIfUnstable()
retryLimit(1)
fixedDelay(600)
}
slackNotifier {
room('#channel')
notifyAborted(true)
notifyFailure(true)
notifyNotBuilt(false)
notifyBackToNormal(true)
notifySuccess(false)
notifyRepeatedFailure(false)
notifyUnstable(true)
startNotification(false)
includeTestSummary(false)
includeCustomMessage(false)
customMessage(null)
sendAs(null)
commitInfoChoice('NONE')
teamDomain(null)
authToken(null)
}
wsCleanup {
cleanWhenAborted(true)
cleanWhenNotBuilt(true)
cleanWhenFailure(true)
cleanWhenSuccess(true)
cleanWhenUnstable(true)
}
}
}
}
The initial error was this (when using Jenkins 2.140):
[EnvInject] - Loading node environment variables.
Building remotely on master-host (docker master) in workspace /home/jenkins/master-host/workspace/seed-job
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/myrepo # timeout=10
Fetching upstream changes from https://github.com/myrepo/jenkins-config
> git --version # timeout=10
using GIT_ASKPASS to set credentials Github
> git fetch --tags --progress https://github.com/myrepo/jenkins-config +refs/heads/*:refs/remotes/origin/*
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision ceb7869d4c6f3c68966e2078afc238bc179cf485 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ceb7869d4c6f3c68966e2078afc238bc179cf485
Commit message: "Appending seperator to workspace path to fix groovy.util.ResourceException"
> git rev-list --no-walk c584afcbfed911d5b965208d78ee04504ffd5ea5 # timeout=10
Processing DSL script jobBuilder.groovy
Processing DSL script jobBuilder.groovy
Processing DSL script jobBuilder.groovy
Processing DSL script jobBuilder.groovy
Processing DSL script jobBuilder.groovy
FATAL: Unable to run script
groovy.util.ResourceException: Cannot open URL: file:/home/jenkins/master-host/workspace/seed-job/jenkins.mydomain.com/config/global_config.groovy
at groovy.util.GroovyScriptEngine.getResourceConnection(GroovyScriptEngine.java:414)
at groovy.util.GroovyScriptEngine.loadScriptByName(GroovyScriptEngine.java:558)
at sun.reflect.GeneratedMethodAccessor3616.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
at jobBuilder$_run_closure1.doCall(jobBuilder.groovy:4)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at groovy.lang.Closure.call(Closure.java:414)
at groovy.lang.Closure.call(Closure.java:430)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.with(DefaultGroovyMethods.java:242)
at org.codehaus.groovy.runtime.dgm$757.invoke(Unknown Source)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at jobBuilder.run(jobBuilder.groovy:3)
at jobBuilder$run.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at jobBuilder$run.call(Unknown Source)
at javaposse.jobdsl.dsl.AbstractDslScriptLoader.runScript(AbstractDslScriptLoader.groovy:132)
at sun.reflect.GeneratedMethodAccessor3088.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:64)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
at javaposse.jobdsl.dsl.AbstractDslScriptLoader.runScriptEngine(AbstractDslScriptLoader.groovy:106)
Caused: java.io.IOException: Unable to run script
at sun.reflect.GeneratedConstructorAccessor711.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:255)
at javaposse.jobdsl.dsl.AbstractDslScriptLoader.runScriptEngine(AbstractDslScriptLoader.groovy:114)
at sun.reflect.GeneratedMethodAccessor3077.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:174)
at javaposse.jobdsl.dsl.AbstractDslScriptLoader$_runScripts_closure1.doCall(AbstractDslScriptLoader.groovy:59)
at sun.reflect.GeneratedMethodAccessor3074.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at groovy.lang.Closure.call(Closure.java:414)
at groovy.lang.Closure.call(Closure.java:430)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2040)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2025)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2078)
at org.codehaus.groovy.runtime.dgm$164.invoke(Unknown Source)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at javaposse.jobdsl.dsl.AbstractDslScriptLoader.runScripts(AbstractDslScriptLoader.groovy:46)
at javaposse.jobdsl.plugin.ExecuteDslScripts.perform(ExecuteDslScripts.java:341)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.Build$BuildExecution.build(Build.java:206)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1815)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
[BFA] Scanning build for known causes...
[BFA] No failure causes found
[BFA] Done. 0s
Started calculate disk usage of build
Finished Calculation of disk usage of build in 0 seconds
Started calculate disk usage of workspace
Finished Calculation of disk usage of workspace in 0 seconds
Finished: FAILURE
And then after updating Jenkins and plugins changed to this:
[EnvInject] - Loading node environment variables.
Building on master in workspace /var/jenkins_home/workspace/seed-job
Cloning the remote Git repository
Cloning repository https://github.com/myrepo/jenkins-config
> git init /var/jenkins_home/workspace/seed-job # timeout=10
Fetching upstream changes from https://github.com/myrepo/jenkins-config
> git --version # timeout=10
using GIT_ASKPASS to set credentials Github
> git fetch --tags --progress https://github.com/myrepo/jenkins-config +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/myrepo/jenkins-config # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/myrepo/jenkins-config # timeout=10
Fetching upstream changes from https://github.com/myrepo/jenkins-config
using GIT_ASKPASS to set credentials Github
> git fetch --tags --progress https://github.com/myrepo/jenkins-config +refs/heads/*:refs/remotes/origin/*
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision asdasdf3c68966e2078afcgth6579cf485 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f asdasdf3c68966e2078afcgth6579cf485
Commit message: "Appending seperator to workspace path to fix groovy.util.ResourceException"
> git rev-list --no-walk asdasdf3c68966e2078afcgth6579cf485 # timeout=10
Processing DSL script jobBuilder.groovy
ERROR: (jobBuilder.groovy, line 67) the following options are required and must be specified: baseUrl
[BFA] Scanning build for known causes...
[BFA] No failure causes found
[BFA] Done. 0s
Started calculate disk usage of build
Finished Calculation of disk usage of build in 0 seconds
Started calculate disk usage of workspace
Finished Calculation of disk usage of workspace in 0 seconds
Finished: FAILURE
So, logically this suggests we need to add the baseUrl to the DSL scripts. But does anyone know why? And does, anyone know why this suddenly changed without anything being changed by us?
EDIT
I've added the following parameter to the scripts (under the slackNotifier) and this has resolved this issue; but it still doesn't answer the question.
baseUrl(null)
This commit in v2.4 of the Slack plugin broke compatibility with Job DSL script, making a lot of options mandatory (again):
Compatibility has been restored in v2.11: