Search code examples
azure-devopsazure-pipelines

How not to fail Azure Pipeline task with CopyFilesOverSSH if files exist and overwrite is false


I have the below CopyFilesOverSSH@0 task in my Azure Pipeline template file. If this is a release build, I want it to copy the build artifacts to an another server using ssh. But only if the server doesn't already have artifacts stored for that build/version. I have overwrite set to false to prevent this.

  - task: CopyFilesOverSSH@0
    displayName: "Copy Release Artifacts to server"
    condition: and(succeeded(), eq(variables.isRelease, '0'))
    inputs:
      sshEndpoint: 'myRemoteServer'
      sourceFolder: "$(Build.ArtifactStagingDirectory)/${{ parameters.buildName }}"
      contents: '**'
      targetFolder: "/artifacts/$(Build.Repository.Name)/${{ parameters.buildName }}/$(buildVersion)"
      readyTimeout: '20000'
      overwrite: false
      failOnEmptySource: true

This task works just fine the first time for a particular build/version, but if the build is repeated, this build task fails because the files already exist. I receive an error similar to:

##[error]Failed to copy /home/azure-pipeline-agents/hostOS_1/50/a/myBuildName/myBuildArtifactFile. File /artifacts/***/myBuildName/v1.2.3/myBuildArtifactFile cannot be copied to the remote machine because it already exists and the 'Overwrite' option is disabled.

How do I either:

  1. Not have this task fail, which then fails the entire build, because the files already exist on the remote server.
  2. Detect the presence of the files on the remote server in an earlier task so that I can skip this copy task in the condition clause.

UPDATE: I have added continueOnError: true to the task and the build now continues, but the build still shows as having errors. It would be nice to be able to differentiate between a "files exist" issue as being ok and errors like "cannot connect to server" or "source files don't exist" as actual errors.


Solution

  • With continueOnError: true on task CopyFilesOverSSH@0, you can continue to next task if it reports error.

    The task is not supported to differentiate between errors and treat as OK or real error.

    As per your requirement:

    Detect the presence of the files on the remote server in an earlier task so that I can skip this copy task in the condition clause.

    You can add a pre task ssh@0 to detect the files, sample code below:

      - task: SSH@0
        inputs:
          sshEndpoint: 'SSHConn'
          runOptions: 'inline'
          inline: |
            if [ -f /pathonremote/myBuildArtifactFile ]; then
              echo "##vso[task.setvariable variable=fileExists;isOutput=true]true"
              echo "##vso[task.logissue type=warning]Release files already exist"
            else
              echo "##vso[task.setvariable variable=fileExists;isOutput=true]false"
            fi
          readyTimeout: '20000'
        name: CheckFile
    

    conditionally run the CopyFilesOverSSH@0 task:

    - task: CopyFilesOverSSH@0
      condition: ne(variables['CheckFile.fileExists'], 'true')
      inputs:
        sshEndpoint: 'your_ssh_service_connection'
        ...