Search code examples
jenkinsjenkins-pipelinejenkins-pluginsjenkins-groovy

Run job on all existing Jenkins workers


I have a Job in a pipeline that cleans up docker images. It runs the job on each worker individually. This is frustrating because when I add jenkins-cpu-worker3, I'll have to update this job. I'd like to run this job in such a way that it runs on all workers without having to update it each time a new worker is present. I also want the job to be able to run regardless of what I name each worker. It needs to run on all workers no matter what. Is there a way to query jenkins from within the pipeline to get me a list or array of all the workers that exist. I was leafing through documentation and posts online and I have not found a solution that works. If possible I'd like to do this without any additional Jenkins Plugins.

pipeline {
  agent any

  stages {

    stage('Cleanup jenkins-cpu-worker1') {
      agent {
        node {
          label 'jenkins-cpu-worker1'
        }
      }

      steps {
        sh "docker container prune -f"
        sh "docker image prune -f"
        sh '''docker images | awk '{print $1 ":" $2}' | xargs docker image rm || true'''
        sh "docker network prune -f"
        sh "docker volume prune -f"
      }
    }

    stage('Cleanup jenkins-cpu-worker2') {
      agent {
        node {
          label 'jenkins-cpu-worker2'
        }
      }

      steps {
        sh "docker container prune -f"
        sh "docker image prune -f"
        sh '''docker images | awk '{print $1 ":" $2}' | xargs docker image rm || true'''
        sh "docker network prune -f"
        sh "docker volume prune -f"
      }
    }

Solution

  • Here is an improved version of your Pipeline. This will dynamically get all the active agents, and run your cleanup task in parallel.

    pipeline {
        agent any
    
        stages {
            stage('CleanupWorkers') {
                steps {
                    script {
                        echo "Something"
                        parallel parallelJobs()
                    }
                }
            }
        }
    }
    
    def parallelJobs() {
      jobs = [:]
      for (def nodeName in getAllNodes()) {
        jobs[nodeName] = getStage(nodeName)
      }
      return jobs
    }
    
    def getStage(def nodeName){
        return { 
            stage("Cleaning $nodeName") {
               node(nodeName){
                    sh'''
                      echo "Srating cleaning"
                      docker container prune -f
                      docker image prune -f
                      docker images | awk '{print $1 ":" $2}' | xargs docker image rm || true
                      docker network prune -f
                      docker volume prune -f
                    '''
               }
             }
        }
    }
    
    def getAllNodes() {
        def nodeNames = [] 
        def jenkinsNodes = Jenkins.instance.getNodes().each { node ->
        // Ignore offline agents
        if (!node.getComputer().isOffline()){
          nodeNames.add(node.getNodeName())
        } 
      }
      return nodeNames
    }