Search code examples
c++protocol-buffersbazelbazel-rules

How can I implement a command line option in Bazel to switch between which dependency version is used for building?

For some background, the C++ program I am working on has the possibility to interoperate with some other applications that use various Protobuf versions. In the source code for my program, I have the compiled .pb.cc files from these other applications for the Protobuf interface. These .pb.cc files were compiled with a particular version of Protobuf, and I don't have any control over this. I am using Bazel to build, and I want to be able to specify a Bazel build configuration for my program, which will use a particular version of Protobuf which matches that of one of the possible other applications.

Originally, I wanted to put something in the .bazelrc file so that I can specify a particular version of Protobuf depending on the config, for example:

# in .bazelrc:
build:my_config --protobuf_version=3_20_1
build:my_other_config --protobuf_version=3_21_6

Then from the terminal, I could build with the command

bazel build --config=my_config //path/to/target:target

which would build as if I had typed

bazel build --protobuf_version=3_20_1 //path/to/target:target

At this point, I wanted to use the select() function, as detailed in the Bazel docs for Configurable Build Attributes, to use a particular Protobuf version during building. But, the Protobuf dependencies are all specified in the WORKSPACE file, which is more limited than a BUILD file, and this select() function cannot be used there. So then my idea was to pull in every version of the Protobuf library that I would possibly need, and give them different names in the WORKSPACE file, and then in the BUILD files, use a select() function to choose the correct version. But, the Bazel rule for compiling the proto_library is used as such:

proto_library(
    name = "foo",
    srcs = ["foo.proto"],
    strip_import_prefix = "/foo/bar/baz",
)

I don't see of any opportunity to use a select() function here to specify which Protobuf version's proto_library rule should be used. The proto_library rule is also defined in from the WORKSPACE file with:

load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()

Now, I would say that I am stuck. I don't see a way to specify on the command line which version of Protobuf should be used with the proto_library rule.

In the end, I would like a way to do the equivalent in the WORKSPACE file of

# in WORKSPACE
if my_config:
    # specific protobuf version:
    http_archive(
      name = "com_google_protobuf",
      sha256 = "8b28fdd45bab62d15db232ec404248901842e5340299a57765e48abe8a80d930",
      strip_prefix = "protobuf-3.20.1",
      urls = ["https://github.com/protocolbuffers/protobuf/archive/v3.20.1.tar.gz"],
    )
elif my_other_config:
    # same as above, but with different version
else:
    # same as above, but with default version

According to some google groups discussion, this doesn't seem to be possible in the WORKSPACE file, so I would need to do it in a BUILD file, but the dependencies are specified in the WORKSPACE.

Solution

  • I figured out a way that works that seems to go against Bazel's philosophy, but most importantly does what I want.

    The repository dependencies are loaded in the first of two steps, the first involving the WORKSPACE file, and the second involving the BUILD file. Command line flags for the build cannot be normally be directly passed to the WORKSPACE, but it is possible to get some information to the WORKSPACE by setting an environment variable and creating a repository_rule. In the WORKSPACE, this environment variable can be used, for example, to change the url argument to http_archive which specifies the dependency version.

    This repository rule is created in a separate file .bzl file, which is then loaded in the WORKSPACE. As a generalized example of how get environment variable values into the WORKSPACE, the following file my_repository_rule.bzl could be created:

    # in file my_repository_rule.bzl
def _my_repository_rule_impl(repository_ctx):
    # read the particular environment variable we are interested in
    config = repository_ctx.os.environ.get("MY_CONFIG_ENV_VAR", "")

    # necessary to create empty BUILD file for this rule
    # which will be located somewhere in the Bazel build files
    repository_ctx.file("BUILD")

    # some logic to do something based on the value of the environment variable passed in:
    if config.lower() == "example_config_1":
        ADDITIONAL_INFO = "foo"
    elif config.lower() == "example_config_2":
        ADDITIONAL_INFO = "bar"
    else:
        ADDITIONAL_INFO = "baz"

    # create a temporary file called config.bzl to be loaded into WORKSPACE
    # passing in any desired information from this rule implementation
    repository_ctx.file("config.bzl", content = """
MY_CONFIG = {}
ADDITIONAL_INFO = {}
""".format(repr(config), repr(ADDITIONAL_INFO ))
    )


my_repository_rule = repository_rule(
    implementation=_my_repository_rule_impl,
    environ = ["MY_CONFIG_ENV_VAR"]
)

    This can be used in the WORKSPACE as such:

    # in file WORKSPACE
load("//:my_repository_rule.bzl", "my_repository_rule ")
my_repository_rule(name = "local_my_repository_rule ")
load("@local_my_repository_rule //:config.bzl", "MY_CONFIG", "ADDITIONAL_INFO")

print("MY_CONFIG = {}".format(MY_CONFIG))
print("ADDITIONAL_INFO = {}".format(ADDITIONAL_INFO))

    When a target is built with bazel build, the WORKSPACE will receive the value of the MY_CONFIG_ENV_VAR from the terminal and store it in the Starlark variable MY_CONFIG, and any other additional information determined in the implementation.

    The environment variable can be passed by normal means, such as typing in a bash shell, for example:

    MY_CONFIG_ENV_VAR=example_config_1 bazel build //path/to/target:target

    It can also be passed as a flag with the --repo_env flag. This flag sends an extra environment variable to be available to the repository rules, meaning the following is equivalent:

    bazel build --repo_env=MY_CONFIG_ENV_VAR=example_config_1 //path/to/target:target

    This can be made easier to switch between by including the following in the .bazelrc file:

    # in file .bazelrc
build:my_config_1 --repo_env=MY_CONFIG_ENV_VAR=example_config_1
build:my_config_2 --repo_env=MY_CONFIG_ENV_VAR=example_config_2

    So running bazel build --config=my_config_1 //path/to/target:target will show the debug output from the print statements in WORKSPACE as the following:

    MY_CONFIG = example_config_1
ADDITIONAL_INFO = foo

    If ADDITIONAL_INFO in the rule implementation (in the file my_repository_rule.bzl) were set to a version number such as "3.20.1", then the WORKSPACE could, for example, use this in an http_archive call to pull the desired version of the dependency.

    # in file WORKSPACE
if ADDITIONAL_INFO == "3.20.1":
    sha256 = "8b28fdd45bab62d15db232ec404248901842e5340299a57765e48abe8a80d930"

http_archive(
  name = "com_google_protobuf",
  sha256 = sha256,
  strip_prefix = "protobuf-{}".format(ADDITIONAL_INFO),
  urls = ["https://github.com/protocolbuffers/protobuf/archive/v{}.tar.gz".format(ADDITIONAL_INFO)],
)

    Of course, the value of the sha256 kwarg could also be passed in from the repository rule as a separate string variable, or as part of a dictionary, for example.