Azure Devops and uploading a Python package to Artifacts using pipeline | authentication issue

Question

I am trying to use the Azure pipeline to publish a Python package to Artifact's feed. I could do it from my local machine and upload the package using Twine, but I have an authentication issue in the pipeline.

trigger:
- main

pool:
  vmImage: ubuntu-22.04
variables:
  pip_cache_dir: '$(Pipeline.Workspace)/.pip_cache'

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.10'
    addToPath: true

- bash: |
    python -m venv worker_venv
    source worker_venv/bin/activate
    pip install --upgrade pip
    pip install pipenv
    pipenv requirements > requirements.txt
    pipenv requirements --dev > requirements-dev.txt
    pip install --cache-dir $(pip_cache_dir) -r ./requirements.txt
    pip install --target="./.python_packages/lib/site-packages" --cache-dir $(pip_cache_dir) -r ./requirements.txt
  displayName: 'Install tools'

- script: |
    source worker_venv/bin/activate
    python setup.py sdist bdist_wheel
  displayName: 'Build package'

- task: TwineAuthenticate@1
  inputs:
    artifactFeed: sample-feed-01

- script: |
    source worker_venv/bin/activate
    python -m twine upload --verbose --config-file $(PYPIRC_PATH) --repository-url https://pkgs.dev.azure.com/**company**/Platform/_packaging/sample-feed-01/pypi/upload/ dist/*
  env:
    TWINE_USERNAME: "azure"
    TWINE_PASSWORD: $(PYPI_TOKEN)
  displayName: 'Upload package to Azure Artifacts'

I tried everything, including GPT-4, but the solutions seem wrong or outdated. this is the error:

/usr/bin/bash --noprofile --norc /home/vsts/work/_temp/75d0c60b-0c2a-44e9-be0f-29d838a3b86e.sh
Uploading distributions to 
https://pkgs.dev.azure.com/**company**/Platform/_packaging/sample-feed-01/pypi/up
load/
INFO     dist/**package**.whl (2.6 KB)                 
INFO     dist/**package**.tar.gz (2.6 KB)                           
INFO     username set by command options                                        
INFO     password set by command options                                        
INFO     username: azure                                                        
INFO     password: <hidden>                                                     
Uploading **package**.whl
25l
  0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/5.7 kB • --:-- • ?
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 kB • 00:00 • ?
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 kB • 00:00 • ?
25hINFO     Response from                                                          
         https://pkgs.dev.azure.com/**company**/Platform/_packaging/sample-feed-0
         1/pypi/upload/:                                                        
         401 Unauthorized                                                       
INFO     {"$id":"1","innerException":null,"message":"TF400813: The user        
         'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this
         resource.","typeName":"Microsoft.TeamFoundation.Framework.Server.Unauth
         orizedRequestException,                                                
         Microsoft.TeamFoundation.Framework.Server","typeKey":"UnauthorizedReque
         stException","errorCode":0,"eventId":3000}                             
ERROR    HTTPError: 401 Unauthorized from                                       
         https://pkgs.dev.azure.com/**company**/Platform/_packaging/sample-feed-0
         1/pypi/upload/                                                         
         Unauthorized                                       

I have some doubts about 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' as user name, which I did not obfuscate, and that is what I see in the pipeline. any help would be appreciated

Answer 1

Go the Azure Artifacts feed you want to publish the Python package to, then navigate to Feed Settings > Permissions page to check and ensure the following identities have been assigned with Contributor role at least.

• Project Collection Build Service ({OrgName})
• {ProjectName} Build Service ({OrgName})

For more details, see "Job access tokens".

---

After configuring the permission role, in the pipeline, you can build and publish the Python package like as below.

steps:
- Some steps to install dependences and build the package.

- task: TwineAuthenticate@1
  displayName: 'Twine Authenticate '
  inputs:
    artifactFeed: {feed_name}

- bash: 'twine upload -r {feed_name} --config-file "$(PYPIRC_PATH)" dist/*.whl'
  displayName: 'Publish to Artifacts feed'

With this way, you do not need to provide your PAT/password to authenticate.

For more details, see "Publish Python packages with Azure Pipelines".

---

EDIT:

By default, the pipeline will use one of the following build identities (also as mentioned above) to access the Azure Artifacts feeds and other resources within current organization:

• Project Collection Build Service ({OrgName}): Can access resources with full/partial permissions across projects within current organization by default.

• {ProjectName} Build Service ({OrgName}): Can access resources with full/partial permissions only within the current project by default.

On the Organization Settings (and Project Settings), there are two options:

• Limit job authorization scope to current project for non-release pipelines
• Limit job authorization scope to current project for release pipelines

If the options are enabled, the pipelines use the identity "{ProjectName} Build Service ({OrgName})". If the options are disabled, the identity "Project Collection Build Service ({OrgName})" is used.

The TwineAuthenticate@1 task will use one of above build identities to generate the twine credentials and set the credentials to the environment variable PYPIRC_PATH.

Since the identities might not have the full access to the feed by default, to assign the required permission role to the identities, you need to click the "Add users/groups" button, then on the pop-up window, search for and add the identities with the required roles.

---