Bazel rules_python未将xgboost复制到runfiles目录致导入失败

I’ve run into similar headaches with older XGBoost versions and rules_python before—let’s break down why numpy works but xgboost isn’t showing up in your runfiles, and how to fix it:

1. Double-Check Your WORKSPACE Dependency Setup #

First, make sure you’re properly pulling in xgboost via rules_python’s pip_parse or pip_install (Bazel can’t track manual virtualenv installs, so stick to its dependency system). Your WORKSPACE should have something like:

pip_parse(
    name = "pip_deps",
    requirements = ":requirements.txt",
)
load("@pip_deps//:requirements.bzl", "install_deps")
install_deps()

And your requirements.txt must explicitly list both packages:

numpy==1.14.3
xgboost==0.71

2. Verify Your BUILD Target Dependencies #

For your print_xgboost_version target, ensure you’re directly depending on the xgboost pip package from your parsed dependencies. A common mistake is missing this line, or using the wrong target name. Your BUILD rule should look like:

py_binary(
    name = "print_xgboost_version",
    srcs = ["print_xgboost_version.py"],
    deps = [
        "@pip_deps//numpy",
        "@pip_deps//xgboost",  # Bazel won't include xgboost without this explicit dependency
    ],
)

If that fails, check the auto-generated BUILD.bazel file in @pip_deps//—sometimes xgboost’s target is nested, like @pip_deps//xgboost:xgboost.

3. Manually Add XGBoost to Runfiles (If Auto-Resolution Fails) #

Older XGBoost versions (like 0.71) have a non-standard wheel structure that rules_python might not fully parse. If the above steps don’t work, force-include the full xgboost package directory as a data dependency:

py_binary(
    name = "print_xgboost_version",
    srcs = ["print_xgboost_version.py"],
    deps = [
        "@pip_deps//numpy",
        "@pip_deps//xgboost",
    ],
    data = [
        "@pip_deps//xgboost:pkg",  # Pulls the entire xgboost package into runfiles
    ],
)

The :pkg target is auto-generated by pip_parse and points to the full installed package directory.

4. Try Source Installation for XGBoost #

If the pre-built wheel is causing issues, force pip to install xgboost from source in your pip_parse setup. This helps rules_python properly track all files, though you’ll need cmake and a C compiler installed on your build machine:

pip_parse(
    name = "pip_deps",
    requirements = ":requirements.txt",
    extra_pip_args = ["--no-binary=xgboost"],
)

Why Numpy Works But XGBoost Doesn’t #

Numpy 1.14.3 follows strict Python package conventions—rules_python easily identifies its module structure and includes all necessary files in runfiles. XGBoost 0.71, however, bundles precompiled binaries in a non-standard way, which can slip through rules_python’s automatic dependency scanning.

内容的提问来源于stack exchange，提问作者concentration