I'm trying to build a custom toolchain definition that references tool binaries from downloadable http_archive package.
It seems that if tool_path.path is relative Bazel treats it as relative to where toolchain is defined. Ex: ${execroot}/toolchain.
http_archive package contents on the other hand are found in ${execroot}/external/gcc12
Tool path must be canonical so you can't specify something like ../external/gcc12/bin/gcc
I looked at other custom toolchain definitions such as:
All of them use wrapper shell scripts that live under toolchain definition (so can be refernced by relative path) and call real binaries via external/gcc... which is relative to the execroot.
While this approach works, it can cause problems with rules_foreign_cc (See https://github.com/bazelbuild/bazel/issues/8438) and anyway seems hacky.
https://github.com/bazelbuild/bazel/pull/10967 "Add optional tool_path_origin to Tools in C++ crosstool" was merged in the May of 2020 which seems to allow relative tool paths to be evaluated relative to workspace root. However I don't quite understand how to trigger this behavior.
https://github.com/bazelbuild/bazel/issues/8438 references https://github.com/bazelbuild/bazel/pull/5809 as relevant but it was rejected in the end.
Another relevant discussion can be found at https://github.com/bazelbuild/bazel/issues/7746
Is it possible to reference Bazel toolchain binaries from external package without resorting to wrapper shell scripts?
So as far as I understand this has mostly been implemented now. Though the interface is kind of painful to use and requires a lot of boiler-plate. As you've mentioned this was introduced in https://github.com/bazelbuild/bazel/pull/10967. But there is next to no documentation included on how to use this new feature.
After following the breadcrumbs creating my own toolchains. I've found that there are two entirely separate interfaces for defining your tools. The first you've mentioned is to use the tool_paths attribute. There is still no way around the shell script for now. This approach is rather well documented and I assume it's what you're currently doing.
The second is to use the action_config method combined with the separate tool constructor.
From the docstring;
Describes a tool associated with a crosstool action config.
Args:
path: Location of the tool; Can be absolute path (in case of non hermetic
toolchain), or path relative to the cc_toolchain's package. If this
parameter is set, tool must not be set.
tool: The built-artifact that should be used as this tool. If this is
set, path must not be set.
So this looks something like this;
action_config (
config_name = ACTION_NAMES.cpp_link_executable,
action_name = ACTION_NAMES.cpp_link_executable,
tools = [
# NEW label type tool field introduced in #10967
tool(tool = ctx.executable.my_ld),
],
)
To get this working in your toolchain you need to go through and create an action_config for each of the ACTION_NAMES. Then you pass your list of action_configs to your toolchain_config.
It's worth noting that while there is some extra-boilerplate here there is also extra flexibility in defining which tools get used and when. There are well defined examples of that in the bazel docs.