TensorFlow v2.*编译失败：absl符号查找错误求助

问题背景 #

近一周尝试编译TensorFlow v2.*版本以优化CPU推理性能，编译至约第10000个目标时失败，报错absl符号缺失。该问题在TF 2.10.0/2.11.0/2.12.0版本、对应Bazel 5.1.1/5.3.0/5.3.0版本中均出现，单核心/多核心编译模式均无法规避。

环境信息 #

OS: ubuntu 18.04
compiler: gcc-v9.3.0/v9.4.0
target tensorflow verison: tensorflow v 2.10.0/2.11.0/2.12.0
bazel version: v 5.1.1/v 5.3.0/ v 5.3.0 respectively

编译命令 #

env CC=/home/echeng/GCC-9.3.0/bin/gcc CXX=/home/echeng/GCC-9.3.0/bin/g++ bazel build --local_cpu_resources=1 --config=opt --copt="-Wno-error=maybe-uninitialized" --copt="-Wno-error" //tensorflow:libtensorflow.so --verbose_failures

报错详情 #

ERROR: /home/echeng/tmp/tensorflow/tensorflow/cc/BUILD:673:22: Executing genrule //tensorflow/cc:io_ops_genrule failed: (Exit 127): bash failed: error executing command
  (cd /home/echeng/.cache/bazel/_bazel_echeng/2791709de7ed8b410670694aec878ce9/execroot/org_tensorflow && \
  exec env - \
    LD_LIBRARY_PATH=:/home/echeng/global_files/lib:/home/echeng/global_files/lib \
    PATH=/usr/local/texlive/2022/texmf-dist/scripts/latexindent:/usr/local/texlive/2022/bin/x86_64-linux:/home/echeng/.cargo/bin:/home/echeng/bin:/usr/local/cuda/bin:/home/echeng/global_files/bin/datagrip/DataGrip/bin/:/home/echeng/third_party_utils/diff_so_fancy:/usr/local/texlive/2022/bin/x86_64-linux:/usr/local/texlive/2022/bin/x86_64-linux:/usr/local/texlive/2022/bin/x86_64-linux,:/home/echeng/.local/bin:/home/echeng/.nvm/versions/node/v16.18.0/bin:/with/tetrion/root/bin:/usr/lib/icecc/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/with/tetrion/scripts:/snap/bin:/opt/puppetlabs/bin:/home/echeng/.fzf/bin:/home/echeng/global_files/bin/:/home/echeng/global_files/bin/ \
    PYTHON_BIN_PATH=/home/echeng/anaconda3/envs/lstm/bin/python3 \
    PYTHON_LIB_PATH=/home/echeng/anaconda3/envs/lstm/lib/python3.8/site-packages \
    TF2_BEHAVIOR=1 \
  /bin/bash bazel-out/k8-opt/bin/tensorflow/cc/io_ops_genrule.genrule_script.sh)
# Configuration: 73b22a0b8b02d57837db5cbfd145ac5b2c1678d8161822f496527a1416447639
# Execution platform: @local_execution_config_platform//:platform
bazel-out/k8-opt/bin/tensorflow/cc/ops/io_ops_gen_cc: symbol lookup error: bazel-out/k8-opt/bin/tensorflow/cc/ops/io_ops_gen_cc: undefined symbol: _ZN4absl12lts_2022062318container_internal11kEmptyGroupE
Target //tensorflow:libtensorflow.so failed to build
INFO: Elapsed time: 5.235s, Critical Path: 0.38s
INFO: 30 processes: 30 internal.
FAILED: Build did NOT complete successfully

已排查情况 #

已在bazel-bin/external/com_google_absl/absl/container/_objs/raw_hash_set/raw_hash_set.o中定位到缺失的符号：

$ nm raw_hash_set.o

                 U _GLOBAL_OFFSET_TABLE_
0000000000000000 r .LC0
0000000000000010 r .LC1
0000000000000000 R _ZN4absl12lts_2022062318container_internal11kEmptyGroupE
0000000000000000 W _ZN4absl12lts_2022062318container_internal19find_first_non_fullIvEENS1_8FindInfoEPKNS1_6ctrl_tEmm
0000000000000000 T _ZN4absl12lts_2022062318container_internal21ShouldInsertBackwardsEmPKNS1_6ctrl_tE
0000000000000000 T _ZN4absl12lts_2022062318container_internal37ConvertDeletedToEmptyAndFullToDeletedEPNS1_6ctrl_tEm
0000000000000000 u _ZZN4absl12lts_2022062318container_internal10RandomSeedEvE7counter

解决方法 #

• 调整链接路径：在编译命令中添加--linkopt="-Lbazel-bin/external/com_google_absl"，确保链接器能找到absl的库文件。
• 强制静态链接absl：修改tensorflow/cc/BUILD中io_ops_genrule的规则，添加linkstatic = True属性；或直接在编译命令中加入--config=monolithic参数，强制TensorFlow整体静态链接。
• 清理Bazel缓存：执行bazel clean --expunge彻底清除缓存后重新编译，避免旧缓存文件引发的依赖冲突。
• 添加位置无关代码参数：在编译命令中加入--copt="-fPIC"，确保生成的目标文件使用位置无关代码，解决动态链接时的符号解析问题。
• 锁定absl版本一致性：修改TensorFlow的WORKSPACE文件，锁定absl的commit版本与TF依赖的版本一致，避免版本不匹配导致的符号差异。

内容的提问来源于stack exchange，提问作者cxz