Search code examples
groovyjenkinsjenkins-pluginsjenkins-build-flow

Parallel jobs using build flow plugin with loop in jenkins


I am building a jenkins flow using the build flow plugin that will grab all of the jobs, compare the name to a regex, and if it matches it will trigger a build of the job.
I have this working perfectly:

import jenkins.model.Jenkins
import java.util.regex.*
Pattern myRegex = ~/release_status.*/
for (item in jenkins.model.Jenkins.instance.items) 
{
  if (item.name ==~ myRegex) {
    build( "$item.name" )
  }
}

However this takes a very long time to build all of the matching jobs (right now there are 20 but there could be many more).
I am trying to make this run each job in parallel but I can't figure out the groovy syntax.
I have tried 3 different ways:

import jenkins.model.Jenkins
import java.util.regex.*
Pattern myRegex = ~/release_status.*/
parallel (
  {
    for (item in jenkins.model.Jenkins.instance.items) 
    {
      if (item.name ==~ myRegex) {
        build( "$item.name" )
      }
    }
  }
)

^^This still works, however it works the same way as before. It goes one job at a time and does not build the next until the previous one finishes.

import jenkins.model.Jenkins
import java.util.regex.*
Pattern myRegex = ~/release_status.*/
parallel (
  {
    for (item in jenkins.model.Jenkins.instance.items) 
    {
      if (item.name ==~ myRegex) {
        { build( "$item.name" ) },
      }
    }
  }
)

^^This errors with

Script1.groovy: 9: Ambiguous expression could be either a parameterless closure expression or an isolated open code block;
   solution: Add an explicit closure parameter list, e.g. {it -> ...}, or force it to be treated as an open block by giving it a label, e.g. L:{...} @ line 9, column 9.
           { build( "$item.name" ) },

break

import jenkins.model.Jenkins
import java.util.regex.*
Pattern myRegex = ~/release_status.*/
parallel (
  [
    for (item in jenkins.model.Jenkins.instance.items) 
    {
      if (item.name ==~ myRegex) {
        useless: { build( "$item.name" ) },
      }
    }
  ]
)

break

import jenkins.model.Jenkins
import java.util.regex.*
Pattern myRegex = ~/release_status.*/
parallel (
    for (item in jenkins.model.Jenkins.instance.items) 
    {
      if (item.name ==~ myRegex) {
        { build( "$item.name" ) },
      }
    }
)

Both of the blocks above error with the following:

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 5: unexpected token: for @ line 5, column 5.
       for (item in jenkins.model.Jenkins.instance.items) 

A lot of code here but its fairly simple. Looking around I can't find many good resources on groovy.


Solution

  • parallel takes a list of Closures, so you should be able to use collect to return a list:

    import jenkins.model.Jenkins
    import java.util.regex.*
    
    Pattern myRegex = ~/release_status.*/
    parallel jenkins.model.Jenkins.instance.items.collect { item ->
        { -> 
            if (item.name ==~ myRegex) {
                build( "$item.name" )
            }
        }
    }
    

    An alternative that only returns a Closure if the name passes (rather than a Closure for every item, a lot of which will finish early) is:

    import jenkins.model.Jenkins
    import java.util.regex.*
    
    Pattern myRegex = ~/release_status.*/
    
    parallel Jenkins.instance.items.findAll { item -> item.name ==~ myRegex}
                                   .collect { item -> { -> build("$item.name") } }