Call jobs from template in different parts of Azure Pipeline

Question

I have a template with jobs that I require to run in an Azure DevOps pipeline, the template is the following:

jobs:
- job: shell-script-job
  steps:
  - script: |
      set -x
      env
      pwd
      echo "CONTAINER_APP=$(Build.Repository.Name):$(Build.BuildId)" > artifact.txt
      source ./artifact.txt
      docker save $CONTAINER_APP > container.tar
      ls -la
    displayName: 'Shell Script: APP'

- job: docker-script-job
  steps:
  - script: | 
      echo "CONTAINER_KNEX=$(Build.Repository.Name):$(Build.BuildId)" > artifact-knex.txt
      source ./artifact-knex.txt
      docker save $CONTAINER_KNEX > container-knex.tar
      ls -la 
    displayName: 'Shell Script: knex'

- job: docker-tasks
  steps:
  - task: PublishPipelineArtifact@1
    displayName: 'Publish Pipeline Artifact - container id: knex'
    inputs:
      targetPath: artifact-knex.txt
      artifact: artifact-knex
  - task: PublishPipelineArtifact@1
    displayName: 'Publish Pipeline Artifact - container data: knex'
    inputs:
      targetPath: container-knex.tar
      artifact: container-knex

I was using different templates for each job before but I was requested to put all of them in one single template I need to call each job individually within the pipeline since they run at different times. like this:

 - stage: build_dev
      dependsOn: quality_control
      displayName: Build DEV
      condition: or(and(eq(variables.isDev, true), ne(variables['Build.Reason'], 'PullRequest')),   eq(variables['Build.Reason'], 'Manual'))
      jobs:
      - job: Job_build_dev
        displayName:  Application Build DEV
        steps:
        - checkout: backend-dev
          submodules: true
        - script: git rev-parse HEAD
        - script: ls -la
        - powershell: |
            Write-Host "##vso[build.addbuildtag]dev"
           displayName: 'Build Tag'
    
      ****- template: templates/template_file.yml/shell_script_job****

        - task: PublishPipelineArtifact@1
          displayName: 'Publish Pipeline Artifact - container id'
          inputs:
            targetPath: artifact.txt
            artifact: artifact 

        - task: PublishPipelineArtifact@1
          displayName: 'Publish Pipeline Artifact - container data'
          inputs:
            targetPath: container.tar
            artifact: container
    
       **** - template: templates/template_file.yml/docker-script-job****
    

What I'm trying to do is call each job individually in the pipeline when the job is needed without calling all of them at once, I'm not sure if this is possible to do or I have to find another way to do this.

I've been following the Azure documentation, trying several ways to invoke the jobs but I'm a little lost: text

Answer 1

What I'm trying to do is call each job individually in the pipeline when the job is needed without calling all of them at once

As per your template.yml, you have defined different jobs. But, in your main.yml, you mixed the job and task together.

You need to use same level in main.yml, you can put the task in a new job, or directly put it in template job.

After you fix the level, then you can invoke the jobs as you needed. Sample code as below:

Main.yaml:

pool:
  vmImage: ubuntu-latest

stages:
- stage: build_dev
  jobs:
  - job: test_job1
    steps: 
    - bash: echo test job1

  - template: template3.yml
    parameters:
      jobName: 'Job1'

  - job: test_job2
    steps:
    - bash: echo test job2
  
  - template: template3.yml
    parameters:
      jobName: 'Job2'

  - template: template3.yml
    parameters:
      jobName: 'Job3'

Template.yml content:

# template.yml
parameters:
  - name: 'jobName'  # this is a parameter
    type: string

jobs:
- job: ${{ format('{0}_{1}', parameters.jobName, 1) }}
  condition: eq('${{ parameters.jobName }}', 'Job1')
  steps:
  - script: echo Running Job1

- job: ${{ format('{0}_{1}', parameters.jobName, 2) }}
  condition: eq('${{ parameters.jobName }}', 'Job2')
  steps:
  - script: echo Running Job2

- job: ${{ format('{0}_{1}', parameters.jobName, 3) }}
  condition: eq('${{ parameters.jobName }}', 'Job3')
  steps:
  - script: echo Running Job3

You need to use ${{ format('{0}_{1}', parameters.jobName, 1 }} as jobName in template, differenet number 1-3 to make sure the jobname won't be duplicated.

Execution result: