使用Bazel运行Python应用时遭遇NumPy C扩展ImportError问题

现象 #

通过Bazel运行依赖flask、transformers、numpy的Python项目时，抛出如下错误：

INFO: Analyzed target //inference:run_app (1 packages loaded, 4 targets configured).
INFO: Found 1 target...
Target //inference:run_app up-to-date:
bazel-bin/inference/run_app
INFO: Elapsed time: 1.159s, Critical Path: 0.69s
INFO: 4 processes: 4 internal.
INFO: Build completed successfully, 4 total actions
INFO: Running command line: bazel-bin/inference/run_app
Traceback (most recent call last):
File "/private/var/tmp/_bazel_modit/96245a706bfbcbe234af177b6d166fc2/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/inference/run_app.runfiles/infaas_deps_numpy/site-packages/numpy/core/__init__.py", line 24, in <module>
from . import multiarray
File "/private/var/tmp/_bazel_modit/96245a706bfbcbe234af177b6d166fc2/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/inference/run_app.runfiles/infaas_deps_numpy/site-packages/numpy/core/multiarray.py", line 10, in <module>
from . import overrides
File "/private/var/tmp/_bazel_modit/96245a706bfbcbe234af177b6d166fc2/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/inference/run_app.runfiles/infaas_deps_numpy/site-packages/numpy/core/overrides.py", line 8, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

...

ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.

本地用requirements.txt创建的虚拟环境直接运行应用完全正常。

配置信息 #

WORKSPACE关键配置 #

pip_parse(
   name = "infaas_deps",
   requirements_lock = "//inference:requirements_lock.txt",
   python_interpreter_target = interpreter_3_11,
   download_only = True,  # 核心问题点之一
)

BUILD.bazel关键配置 #

py_library(
    name = "svc",
    deps = [
        requirement("flask"),
        requirement("transformers"),
        requirement("torch"),
        # 缺失numpy依赖声明
    ],
)

原因分析 #

1. download_only=True的影响：该参数会让pip_parse仅下载依赖的源码包，跳过编译、安装步骤。NumPy包含需要编译的C扩展模块，源码包本身没有预编译的_multiarray_umath等共享对象，导致运行时无法找到这些模块；而虚拟环境中pip会自动完成编译安装流程。
2. BUILD文件缺失依赖声明：当前svc库的依赖列表未包含requirement("numpy")，即使pip_parse下载了numpy，Bazel也不会自动将未声明的依赖加入到目标的runfiles中，进一步引发模块查找失败。

解决方案 #

步骤1：修复BUILD文件，添加NumPy依赖 #

修改inference/BUILD.bazel中的svc库，补充numpy依赖：

py_library(
    name = "svc",
    deps = [
        requirement("flask"),
        requirement("transformers"),
        requirement("torch"),
        requirement("numpy"),  # 添加此行
    ],
)

步骤2：移除download_only=True，让Bazel自动构建NumPy扩展 #

修改WORKSPACE中的pip_parse配置，删除download_only参数（默认值为False）：

pip_parse(
   name = "infaas_deps",
   requirements_lock = "//inference:requirements_lock.txt",
   python_interpreter_target = interpreter_3_11,
   # 移除 download_only = True
)

可选：使用预编译轮子加速构建 #

若想避免本地编译耗时，可在requirements_lock.txt中指定对应平台的预编译numpy轮子：

numpy==1.26.4 --only-binary=numpy

这样pip_parse会直接下载预编译好的二进制包，无需本地编译。

验证修复 #

执行以下命令清理缓存并重新运行应用：

bazel clean
bazel run //inference:run_app

内容的提问来源于stack exchange，提问作者ModiCnvrg