如何在Bazel中指定使用特定Python工具链？

问题背景 #

我需要配置Bazel，让它能选择两个Python工具链中的一个：一个是从源码构建、会被打包进可执行.zip的封闭工具链（hermetic），另一个是使用系统Python的非封闭工具链。但目前构建总是优先使用第一个注册的python3_toolchain，就算构建目标导入了hermetic_python3_dependencies的requirement规则也没用。

现有配置代码 #

WORKSPACE 文件 #

# WORKSPACE

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
load("@rules_python//python:pip.bzl", "pip_install")

http_archive(
    name = "rules_python",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.5.0/rules_python-0.5.0.tar.gz",
    sha256 = "cd6730ed53a002c56ce4e2f396ba3b3be262fd7cb68339f0377a45e8227fe332",
)

# 非封闭工具链
register_toolchains("//src:python3_toolchain")

pip_install(
   quiet = False,
   name = "python_dependencies",
   requirements = "//:requirements.txt",
   python_interpreter = "/usr/bin/python3"
)

load("@python_dependencies//:requirements.bzl", "requirement")

# 封闭工具链
_py_configure = """
if [[ "$OSTYPE" == "darwin"* ]]; then
    ./configure --prefix=$(pwd)/bazel_install --with-openssl=$(brew --prefix openssl)
else
    ./configure --prefix=$(pwd)/bazel_install
fi
"""

http_archive(
    name = "hermetic_interpreter",
    urls = ["https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tar.xz"],
    sha256 = "a57dc82d77358617ba65b9841cee1e3b441f386c3789ddc0676eca077f2951c3",
    strip_prefix = "Python-3.11.0",
    patch_cmds = [
        "mkdir $(pwd)/bazel_install",
        _py_configure,
        "make",
        "make install",
        "ln -s bazel_install/bin/python3 python_bin",
    ],
    build_file_content = """
exports_files(["python_bin"])
filegroup(
    name = "files",
    srcs = glob(["bazel_install/**"], exclude = ["**/* *"]),
    visibility = ["//visibility:public"],
)
""",
)

pip_install(
    name = "hermetic_python3_dependencies",
    requirements = "//:requirements.txt",
    python_interpreter_target = "@hermetic_interpreter//:python_bin",
)
load("@hermetic_python3_dependencies//:requirements.bzl", "requirement")

load("@rules_python//python:defs.bzl", "py_binary")
load("@rules_python//python:defs.bzl", "py_library")

register_toolchains("//src:hermetic_python3_toolchain")

src/BUILD 文件 #

# src/BUILD

load("@bazel_tools//tools/python:toolchain.bzl", "py_runtime_pair")

# 非封闭工具链
py_runtime(
    name = "python3_runtime",
    interpreter_path = "/usr/bin/python3",
    python_version = "PY3",
    visibility = ["//visibility:public"],
)

py_runtime_pair(
    name = "python3_runtime_pair",
    py2_runtime = None,
    py3_runtime = ":python3_runtime",
)

toolchain(
    name = "python3_toolchain",
    toolchain = ":python3_runtime_pair",
    toolchain_type = "@bazel_tools//tools/python:toolchain_type",
)

# 封闭工具链
py_runtime(
    name = "hermetic_python3_runtime",
    files = ["@hermetic_interpreter//:files"],
    interpreter = "@hermetic_interpreter//:python_bin",
    python_version = "PY3",
    visibility = ["//visibility:public"],
)

py_runtime_pair(
    name = "hermetic_python3_runtime_pair",
    py2_runtime = None,
    py3_runtime = ":hermetic_python3_runtime",
)

toolchain(
    name = "hermetic_python3_toolchain",
    toolchain = ":hermetic_python3_runtime_pair",
    toolchain_type = "@bazel_tools//tools/python:toolchain_type",
)

package(default_visibility = ["//visibility:public"])

/src/some_tool/BUILD 文件 #

# /src/some_tool/BUILD

load("@hermetic_python3_dependencies//:requirements.bzl", "requirement") # 从两个依赖仓库导入都没区别

py_binary(
    name = "some-tool",
    main = "some_tool.py",
    srcs = ["some_tool_file.py"],
    python_version = "PY3",
    srcs_version = "PY3",
    deps = [
        requirement("requests"),
        "//src/common/some-library:library",
    ]
)

package(default_visibility = ["//visibility:public"])

解决方案 #

Bazel的工具链选择逻辑不依赖于你导入的pip仓库，而是基于工具链注册顺序和约束匹配。要实现切换，有两种可靠方式：

方式1：命令行指定工具链 #

直接在构建命令中用--toolchain参数指定要使用的工具链标签：

• 使用非封闭工具链：

  bazel build //src/some_tool:some-tool --toolchain=//src:python3_toolchain
  

• 使用封闭工具链：

  bazel build //src/some_tool:some-tool --toolchain=//src:hermetic_python3_toolchain
  

方式2：目标级强制指定工具链 #

给特定的py_binary或py_library添加toolchains属性，绑定到目标工具链：
修改/src/some_tool/BUILD中的py_binary：

py_binary(
    name = "some-tool",
    main = "some_tool.py",
    srcs = ["some_tool_file.py"],
    python_version = "PY3",
    srcs_version = "PY3",
    deps = [
        requirement("requests"),
        "//src/common/some-library:library",
    ],
    # 强制使用封闭工具链
    toolchains = ["//src:hermetic_python3_toolchain"],
)

如果需要默认用非封闭工具链，其他目标单独指定封闭工具链，这种方式最灵活。

进阶：通过约束条件自动选择（可选） #

如果想让Bazel根据平台或自定义约束自动选择工具链，可以给工具链添加constraint_values：

1. 先定义自定义约束（在src/BUILD中）：

constraint_setting(name = "python_toolchain_type")
constraint_value(
    name = "hermetic_python",
    constraint_setting = ":python_toolchain_type",
)
constraint_value(
    name = "system_python",
    constraint_setting = ":python_toolchain_type",
)

2. 修改工具链定义，添加约束：

# 非封闭工具链
toolchain(
    name = "python3_toolchain",
    toolchain = ":python3_runtime_pair",
    toolchain_type = "@bazel_tools//tools/python:toolchain_type",
    target_compatible_with = [":system_python"],
)

# 封闭工具链
toolchain(
    name = "hermetic_python3_toolchain",
    toolchain = ":hermetic_python3_runtime_pair",
    toolchain_type = "@bazel_tools//tools/python:toolchain_type",
    target_compatible_with = [":hermetic_python"],
)

3. 构建时通过--platforms指定约束：

# 使用封闭工具链
bazel build //src/some_tool:some-tool --platforms=//src:hermetic_python_platform

（需要先定义对应的platform目标）

注意事项 #

• 导入不同pip仓库的requirement只是依赖安装时用不同Python，但构建时的运行时还是由工具链决定，两者要对应起来，否则可能出现依赖不兼容的问题。
• 如果使用目标级toolchains属性，优先级高于命令行和注册顺序，适合固定要用特定工具链的目标。

内容的提问来源于stack exchange，提问作者user17648384