Azure DevOps Pipelines - .artifactignore not working properly with pipeline artifact
I need once more your swarm intelligence with this issue.
Background: I publish powershell modules to azure artifacts using azure pipelines (like this https://learn.microsoft.com/en-us/azure/devops/artifacts/tutorials/private-powershell-library?view=azure-devops&tabs=windows). I have different files (Folder: build_scripts) in my repository that i do not want to publish to azure artifacts. So i came to .artifactignore
Current situation: No mather what i try the files in the folder "build_scripts" stays part of the artifact.
Let me show: My Repository:
Pipeline yaml:
- main
name: 'PowerShell Module Project'
variables:
major: 1
minor: 0
patch: $(Build.BuildID)
buildVer: $(major).$(minor).$(Build.BuildID)
pool: DevPool
stages:
- stage: Build
jobs:
- job: Build
steps:
- task: PowerShell@2
inputs:
filePath: '$(System.DefaultWorkingDirectory)/build_scripts/build.ps1'
- task: NuGetCommand@2
inputs:
command: 'pack'
packagesToPack: '$(System.DefaultWorkingDirectory)/PowerShellModuleProject.nuspec'
versioningScheme: byEnvVar
versionEnvVar: buildVer
buildProperties: 'VERSIONHERE=$(buildVer)'
- task: CopyFiles@2
displayName: 'Copy .artifactignore'
inputs:
SourceFolder: '$(build.sourcesdirectory)'
Contents: '**/.artifactignore'
TargetFolder: '$(build.artifactstagingdirectory)'
flattenFolders: true
- task: PublishPipelineArtifact@1
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'
artifactName: 'NuGetPackage'
publishLocation: 'pipeline'
- stage: Deploy
jobs:
- job: Deploy
steps:
- task: DownloadPipelineArtifact@2
inputs:
buildType: 'current'
artifactName: 'NuGetPackage'
itemPattern: '**'
targetPath: '$(Pipeline.Workspace)'
- task: NuGetCommand@2
inputs:
command: 'push'
packagesToPush: '$(Pipeline.Workspace)/**/*$(buildVer).nupkg'
nuGetFeedType: 'internal'
publishVstsFeed: 'MYProjectName/MyPipelineName'
My .artifactignore file:
build_scripts/**
The artifactignore file is in the correct directory using the copy job, like suggested here on stack overflow (PublishPipelineArtifact .artifact ignore not recognized). This can also be verified when i take a look at the pipeline run log:
Using .artifactignore file located at: C:\agent\vsts-agent-win-x64-3.232.0\_work\3\a\.artifactignore for globbing
2 files processed.
But at the end, when i install the powershell module from the repository, the files in the build_scripts are still present
Now i ran out of ideas. Does anyone have a hint what i am doing wrong here ?
Thanks in advance
Requested additional information from Kevin Lu-MSFT:
Screenshot of the artifact content:
Pipeline log -> Publish Pipeline artifacts:
Starting: PublishPipelineArtifact
==============================================================================
Task : Publish Pipeline Artifacts
Description : Publish (upload) a file or directory as a named artifact for the current run
Version : 1.199.0
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/publish-pipeline-artifact
==============================================================================
Artifact name input: NuGetPackage
Uploading pipeline artifact from C:\agent\vsts-agent-win-x64-3.232.0\_work\3\a for build #36
Using default max parallelism.
Max dedup parallelism: 192
DomainId: 0
ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session a79f1780-5577-434e-9f9e-3f5e8b41c6ab
Hashtype: Dedup1024K
DedupManifestArtifactClient will correlate http requests with X-TFS-Session a79f1780-5577-434e-9f9e-3f5e8b41c6ab
Using .artifactignore file located at: C:\agent\vsts-agent-win-x64-3.232.0\_work\3\a\.artifactignore for globbing
2 files processed.
Processed 2 files from C:\agent\vsts-agent-win-x64-3.232.0\_work\3\a successfully.
Uploaded 48,185 out of 48,185 bytes
Content upload is done!
Content upload statistics:
Total Content: 96.0 KB
Physical Content Uploaded: 41.4 KB
Logical Content Uploaded: 48.2 KB
Compression Saved: 6.8 KB
Deduplication Saved: 47.9 KB
Number of Chunks Uploaded: 3
Total Number of Chunks: 5
Associated artifact 22 with build 36
ApplicationInsightsTelemetrySender correlated 2 events with X-TFS-Session a79f1780-5577-434e-9f9e-3f5e8b41c6ab
Uploading pipeline artifact finished.
Finishing: PublishPipelineArtifact
From your screenshot, the build_scripts folder doesn't exist in the Pipeline Artifacts content.
This means that the build_scripts folder has been added to the nuget package when running the Nuget Pack task.
In this case, when you download the nuget package, it will still contains the build_scripts folder.
Refer to this doc: Use .artifactignore
The .artifactignore is a text file that controls which files are uploaded when you publish a Universal Package or a Pipeline Artifact.
So the .artifactignore will not work when running the nuget pack task.
To solve this issue, you can refer to the following methods:
Method1: you can add the -Exclude argument in the Nuget pack task to exclude the build_scripts folder.
For example:
name: 'PowerShell Module Project'
variables:
major: 1
minor: 0
patch: $(Build.BuildID)
buildVer: $(major).$(minor).$(Build.BuildID)
pool: DevPool
stages:
- stage: Build
jobs:
- job: Build
steps:
- task: PowerShell@2
inputs:
filePath: '$(System.DefaultWorkingDirectory)/build_scripts/build.ps1'
- task: NuGetCommand@2
displayName: 'NuGet custom'
inputs:
command: custom
arguments: 'pack $(System.DefaultWorkingDirectory)/PowerShellModuleProject.nuspec -NonInteractive -OutputDirectory $(Build.ArtifactStagingDirectory) -Properties VERSIONHERE=$(buildVer) -version $(buildVer) -Exclude "build_scripts/*"'
- task: CopyFiles@2
displayName: 'Copy .artifactignore'
inputs:
SourceFolder: '$(build.sourcesdirectory)'
Contents: '**/.artifactignore'
TargetFolder: '$(build.artifactstagingdirectory)'
flattenFolders: true
- task: PublishPipelineArtifact@1
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'
artifactName: 'NuGetPackage'
publishLocation: 'pipeline'
- stage: Deploy
jobs:
- job: Deploy
steps:
- task: DownloadPipelineArtifact@2
inputs:
buildType: 'current'
artifactName: 'NuGetPackage'
itemPattern: '**'
targetPath: '$(Pipeline.Workspace)'
- task: NuGetCommand@2
inputs:
command: 'push'
packagesToPush: '$(Pipeline.Workspace)/**/*$(buildVer).nupkg'
nuGetFeedType: 'internal'
publishVstsFeed: 'MYProjectName/MyPipelineName'
Method2: you can use the nuget pack command in the deploy stage. And in the build stage, you can publish all files to pipelines artifacts(use .artifactignore to exclude build_script file).
For example:
name: 'PowerShell Module Project'
variables:
major: 1
minor: 0
patch: $(Build.BuildID)
buildVer: $(major).$(minor).$(Build.BuildID)
pool: DevPool
stages:
- stage: Build
jobs:
- job: Build
steps:
- task: PowerShell@2
inputs:
filePath: '$(System.DefaultWorkingDirectory)/build_scripts/build.ps1'
- task: PublishPipelineArtifact@1
inputs:
targetPath: '$(Build.sourcesdirectory)'
artifactName: 'NuGetPackage'
publishLocation: 'pipeline'
- stage: Deploy
jobs:
- job: Deploy
steps:
- task: DownloadPipelineArtifact@2
inputs:
buildType: 'current'
artifactName: 'NuGetPackage'
itemPattern: '**'
targetPath: '$(Pipeline.Workspace)'
- task: NuGetCommand@2
inputs:
command: 'pack'
packagesToPack: '$(Pipeline.Workspace)/PowerShellModuleProject.nuspec'
versioningScheme: byEnvVar
versionEnvVar: buildVer
buildProperties: 'VERSIONHERE=$(buildVer)'
packDestination: $(Pipeline.Workspace)
- task: NuGetCommand@2
inputs:
command: 'push'
packagesToPush: '$(Pipeline.Workspace)/*$(buildVer).nupkg'
nuGetFeedType: 'internal'
publishVstsFeed: 'MYProjectName/MyPipelineName'