Search code examples
azure-machine-learning-service

How to specify pip extra-index-url when creating an azureml environment?


When AzureML creates a python environment and runs pip install, I'd like it to use additional non-public indices. Is there a way to do that?

I'm running my python script on an AzureML compute. The environment is created from pip requirements as per docs. The script now references a package in a private index. To run the script on a local or build machine I just specify PIP_EXTRA_INDEX_URL environment variable with credentials to the index before running pip install -c .... How to enable same functionality on AzureML environment prep process?

AzureML docs suggest that I directly supply wheel files instead of package names. That means I have to manually do all the work that pip is built for: identify private packages among other requirements, choose right versions and platform, download them.

Ideally, I would have to just write something like this:

myenv = Environment.from_pip_requirements(
    name = "myenv",
    file_path = "path-to-pip-requirements-file",
    extra-index-url = ["url1", "url2"])

Solution

  • It appears, there is a set_pip_option method in the SDK which sorts out the problem with one single extra-index-url, e.g.

    from azureml.core.environment import CondaDependencies
    dep = CondaDependencies.create(pip_packages=["pyyaml", "param"])
    dep.set_pip_option("--extra-index-url https://user:password@extra.index/url")
    

    Unfortunately, second call to this function replaces the first value with the new one. For the --extra-index-url option this logic should be changed in order to support search in more than 2 indices (one public, one private).