Conditional Execution of a Task in CICD pipeline Job not getting Executed

Question

I have the below main cicd-pipeline.yaml file which reads the latest files from our repo and prepares it to be deployed to our Dev workspace. I am using a Powershell task in the job to get the modified files from git for the current commit and then saving it to a temp location before proceeding to push it to the Build.ArtifactsStagingDirectory using the CopyFiles@2 task, before my DeployNotebooks stage could utilize those files from that location to be pushed to the databricks workspace. But I want this CopyFiles@2 task to be run only when the Powershell task is success and the flag inside it is set to True.

trigger:
  branches:
    include:
      - 'main'

variables:
  - group: dbw-cicd-dev
  
  - name: vmImageName
    value: "windows-latest"
  - name: notebooksPath
    value: "notebooks"
  - name: notebooksFolder
    value: false

pool:
  vmImage: $(vmImageName)

stages:
  - stage: ChangedFolder
    displayName: "Checking the Notebooks folder changes"
    jobs:
      - job: preCheck
        displayName: "Copy_Notebooks"
        steps:
          - checkout: self
            fetchDepth: 2

          - powershell: |
              $files=$(git diff HEAD HEAD~ --name-only)
              $temp=$files -split ' '
              $count=$temp.Length
              echo "Total number of changed files: $count"
              For ($i=0; $i -lt $count; $i++)
              {
                $name=$temp[$i]
                echo "Changed file is: $name"
                if ($name -like 'notebooks/*')
                {
                  $flag = 'True'
                  echo "Is it a notebooks change: $flag"
                  Write-Host "##vso[task.setvariable variable=notebooksFolder;isOutput=true]True"
                }
              }
              ##echo ${{variables.notebooksPath}}
              if (($flag -eq 'True'))
              {
                ## Now create a temp folder to hold the changed files
                New-Item -ItemType directory -Path $(system.defaultworkingdirectory)\temp

                foreach($file in $temp)
                {
                  if(Test-Path -path $file)
                  {
                    Copy-Item -Path $file -Destination $(system.defaultworkingdirectory)\temp
                    echo "File Moved: $file"
                  }
                }
              }              
            name: taskVariable

          - task: CopyFiles@2
            inputs:
              SourceFolder: '$(system.defaultworkingdirectory)\temp'
              Contents: '*.py'
              TargetFolder: '$(Build.ArtifactStagingDirectory)/notebooks'
              CleanTargetFolder: true
            condition: and(succeeded(), eq(variables.notebooksFolder, True))

  - stage: DeployNotebooks
    displayName: "Deploy to DEV Environment"
    dependsOn: ChangedFolder
    condition: |
      and( 
        succeeded('ChangedFolder'),
        eq(dependencies.changedFolder.outputs['precheck.taskVariable.notebooksFolder'], 'True')
      )
    jobs:
      - job: Deploy_Notebooks
        steps:
          - template: templates/deploy-notebooks.yml
            parameters:              
              environmentName: $(dev-environment-name)
              resourceGroupName: $(dev-resource-group-name)
              serviceConnection: $(dev-service-connection-name)
              notebooksPath: $(notebooksPath)

But whenever the pipeline is run, this task is being skipped on evaluation.

Could someone look into the condition block of CopyFiles@2 task and let me know what exactly am I missing here.

But when I change the condition to OR for the CopyFiles@2 task, it runs and errors out as below:

Answer 1

you are missing the name of the task from where the variable should come. See this example:

- task: PowerShell@2
  name: setVariable
  displayName: "Set Variable"
  inputs:
    pwsh: true
    targetType: 'inline'
    script: |
      Write-Host "##vso[task.setvariable variable=notebooksFolder;isOutput=true]True"

- task: PowerShell@2
  condition: and(succeeded(), eq(variables['setVariable.notebooksFolder'], True))
  displayName: "Var condition"
  inputs:
    pwsh: true
    targetType: 'inline'
    script: |
      Write-Host "notebooksFolder: $(setVariable.notebooksFolder)"

Note the name of the first powershell task is included in the condition of the second.

The example above will give you an output like this: