Search code examples
jenkinsjenkins-pipelinedevopsjenkins-declarative-pipeline

How to build a combination of parallel and sequential stages in Jenkins pipeline with dynamic data


I am trying to build a Jenkins pipeline which has a combination of parallel and sequential stages. I am able to accomplish the same with static data but failing to get it working when using dynamic data, i.e. when using a parameterized build and reading data from the build parameters.

Below snippet works fine

pipeline {
agent any
stages {
    stage('Parallel Tests') {
        parallel {
            stage('Ordered Tests Set') {
                stages {
                    stage('Building seq test 1') {
                        steps {
                             echo "build seq test 1"
                            }
                    }
                    stage('Building seq test 2') {
                        steps {
                               echo "build seq test 2"
                        }
                    }
                    
                }
            }

            stage('Building Parallel test 1') {
                steps {
                    echo "Building Parallel test 1"
                }
            }
            stage('Building Parallel test 2') {
                steps {
                   echo "Building Parallel test 2"
                }
            }   
        }

    }
}

}

Gives me the following execution result enter image description here

Now i want to read the values from my build parameters and just loop the stages . This is what i have tried but could not get it to work. This bit of snippet is taken from another answer i found few months back in SO but unable to trace now, else would have added the link -

def parallelStagesMap = params['Parallel Job Set'].split(',').collectEntries {
    ["${it}" : generateStage(it)]
}

def orderedStagesMap = params['Ordered Job Set'].split(',').collectEntries {
    ["${it}" : generateStage(it)]
}

def orderedMap (){
    def orderedStagesMapList= [:]
    orderedStagesMapList['Ordered Tests Set']= {
        stage('Ordered Tests Set') {
            stages{
                orderedStagesMap
            }        
        }
    }
    return orderedStagesMapList;
}

def generateStage(job) {
    return {
        stage("stage: ${job}") {
            echo "This is ${job}."
        }
    }
}

pipeline {
    agent none
    stages {
        stage ("Parallel Stage to trigger Tests"){
            steps {
                script {
                    parallel orderedMap()+parallelStagesMap
                }
            }
        }
    }
}

Solution

  • Declarative and Scripted Pipeline syntax do not mix in Pipeline, see Pipeline Syntax. Since you are dynamically creating a Pipeline definition based on the parameters, you should most likely go completely to Scripted Syntax, unless your use-case matches matrix.

    Removing the Declarative syntax from your Pipeline Definition would give something like below. Note that I did not test it on the live Jenkins instance.

    def parallelStagesMap = params['Parallel Job Set'].split(',').collectEntries {
        ["${it}" : generateStage(it)]
    }
    
    def orderedStagesMap = params['Ordered Job Set'].split(',').collectEntries {
        ["${it}" : generateStage(it)]
    }
    
    def orderedMap (){
        def orderedStagesMapList= [:]
        orderedStagesMapList['Ordered Tests Set']= {
            stage('Ordered Tests Set') {
                orderedStagesMap.each { key, value ->
                    value.call()
                }
            }
        }
        return orderedStagesMapList;
    }
    
    def generateStage(job) {
        return {
            stage("stage: ${job}") {
                echo "This is ${job}."
            }
        }
    }
    
    stage("Parallel Stage to trigger Tests") {
        parallel orderedMap()+parallelStagesMap
    }