Search code examples
pythonbazelpypi

Is it possible to change the index URL for fetching `rules_python` itself in bazel?

I am currently attempting to do a custom build of envoy on a machine that does not have access to PyPi. My company's security team requires us to use a corporate proxy with a different URL to access the PyPi repos.

When I try to run bazel build ..., I get the following error:

ERROR: An error occurred during the fetch of repository 'pypi__pip_tools':
   Traceback (most recent call last):
        File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/b4e0fd0e207e6fdf5e33997b6741cf2d/external/bazel_tools/tools/build_defs/repo/http.bzl", line 132, column 45, in _http_archive_impl
                download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://files.pythonhosted.org/packages/0d/dc/38f4ce065e92c66f058ea7a368a9c5de4e702272b479c0992059f7693941/pip_tools-7.4.1-py3-none-any.whl] to /home/ubuntu/.cache/bazel/_bazel_ubuntu/b4e0fd0e207e6fdf5e33997b6741cf2d/external/pypi__pip_tools/temp12252431299166532778/pip_tools-7.4.1-py3-none-any.whl.zip: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The security error is a red herring in this case because I need to replace https://files.pythonhosted.org with a different domain.

For command line pip, I had to do the following:

python3 -m pip config set global.index-url  <new_url_here>

Is there some equivalent way to force bazel to use a different domain when fetching Python tools?

Note that I have already seen this issue, but it does not help because this error is happening as part of the process of installing pip itself, before any pip_install call is made.

Solution

  • If I understood your question correctly, you may want to check out --experimental_downloader_config flag which allows you to manipulate (bazel's builtin) downloader behavior.

    For instance with downloader config file:

    rewrite ^(files.pythonhosted.org/.*) https://your.host/mirrors/$1

    Which would try to get file in your example:

    https://files.pythonhosted.org/packages/0d/dc/38f4ce065e92c66f058ea7a368a9c5de4e702272b479c0992059f7693941/pip_tools-7.4.1-py3-none-any.whl

    from

    https://your.host/mirrors/files.pythonhosted.org/packages/0d/dc/38f4ce065e92c66f058ea7a368a9c5de4e702272b479c0992059f7693941/pip_tools-7.4.1-py3-none-any.whl

    instead.