Search code examples
azureazure-devopsazure-artifacts

How to Bulk Download Azure Artifacts Packages?

My organization is moving our software packages (NuGet, NPM) to Azure Artifacts. We have approximately 50,000 NuGet packages (2GB) that we've pushed to Azure Artifacts.

Is there a way to to bulk-pull my packages from Azure Artifacts either to my machine or to Azure blob storage in case we are not satisfied with the product? I'm mainly concerned about pulling our NuGet packages because we have thousands of packages published.

I'm aware of Download all packages from private nuget feed but I'd like to know if there's a way to retrieve a zip file of all of our packages, or some other method to download all of our packages in bulk.

Thanks!

Solution

  • We don't have a bulk download option, but it would be pretty easy to script something yourself.

    First, you'll want to get a PAT. Make sure the PAT has the Packaging (read) scope. You will use the PAT as a password for authentication. The username you use doesn't matter.

    Then, you'll need to get the list of all packages in the feed. You can use our Get Packages API for that.

    Then, for NuGet packages, the protocol is as follows:

    1. Download the feed URL (from the Connect to Feed dialog, will end with .../v3/index.json)
    2. Read the returned JSON document, get the @id attribute from the object with @type = PackageBaseAddress/3.0.0. This is the PackageBaseAddress. Note that the actual value is considered an implementation detail. Always retrieve the address from index.json.
    3. If you do not already have the list of versions, download {PackageBaseAddress}/{PackageName (lowercased)}/index.json which will have the list as a JSON array (I suggest getting the versions this way since they will already be normalized)
    4. Download the packages from {PackageBaseAddress}/{PackageName (lowercase)}/{PackageVersion (normalized)}/{PackageName (lowercase)}.{PackageVersion (normalized)}.nupkg
      • e.g. {PackageBaseAddress}/restsharp/106.4.1/restsharp.106.4.1.nupkg
      • Be sure to follow redirects, as the service will actually send you to download from our backend storage

    For npm packages:

    1. Get the Feed URL from the Connect to Feed dialog (it will end with .../npm/registry)
    2. Download {Feed URL}/{PackageName} (for scoped packages include the scope, e.g. {Feed URL}/@{Scope}/{PackageName})
    3. Download URLs for each version are listed in the JSON response as versions.*.dist.tarball
    4. Again, be sure to handle redirects when downloading the packages.