Search code examples
loopsjenkinsgroovyconcurrent-processing

Jenkins - How to Create Dynamic Parallel Stages?


I basically have 4 curl commands to trigger a webhook and they're roughly the same except for a single argument. Therefore, I would like to extract this curl command and create a parallel stage that can iterate through each unique argument and then call the curl command.

I have something like this:

 stage('Push Artifactory Images to ECR') {
                    steps {
                        script {
                            def images = [
                                ['stageName': 'Push Metadata Service Image', 'image': env.METADATA_SERVICE_IMAGE],
                                ['stageName': 'Push UI Image', 'image': env.UI_IMAGE],
                                ['stageName': 'Push AWS Batch Default Image', 'image': env.BATCH_IMAGE],
                                ['stageName': 'Push Studio Sample Image', 'image': env.NOTEBOOK_IMAGE]
                            ]

                            def parallelStages = [:]

                            images.each { image ->
                                parallelStages[image.stageName] = {
                                    stage(image.stageName) {
                                        steps {
                                            script {
                                                sh """
                                                curl ${env.ECR_PUSH_URL} -X POST -H 'Content-Type: application/json' \
                                                -d '{"parameters":{"aws_account_id": "${env.AWS_ACCOUNT_ID}", "business_service": "${env.BUSINESS_SERVICE}", "moniker": "${env.MONIKER}", "region": "${env.REGION}", "role_arn": "arn:aws:iam::${env.AWS_ACCOUNT_ID}:role/Deployer", "sourceRepository": "${image.image}"}}'
                                                """
                                            }
                                        }
                                    }
                                }
                            }

But this produces the following error:

Also:   java.lang.NoSuchMethodError: No such DSL method 'steps' found among steps 

Solution

  • approximately like this:

    def imageStage(image) {
        steps {
            script {
                sh """
                curl ${env.ECR_PUSH_URL} -X POST -H 'Content-Type: application/json' \
                -d '{"parameters":{"aws_account_id": "${env.AWS_ACCOUNT_ID}", "business_service": "${env.BUSINESS_SERVICE}", "moniker": "${env.MONIKER}", "region": "${env.REGION}", "role_arn": "arn:aws:iam::${env.AWS_ACCOUNT_ID}:role/Deployer", "sourceRepository": "${image.image}"}}'
                """
            }
        }
    }
    
    def images = [
        ['stageName': 'Push Metadata Service Image', 'image': env.METADATA_SERVICE_IMAGE],
        ['stageName': 'Push UI Image', 'image': env.UI_IMAGE],
        ['stageName': 'Push AWS Batch Default Image', 'image': env.BATCH_IMAGE],
        ['stageName': 'Push Studio Sample Image', 'image': env.NOTEBOOK_IMAGE]
    ]
    
    def parallelStages = images.collectEntries{ image->
        [ image.stageName, { imageStage(image) } ]
    }
    
    parallel parallelStages