TensorFlow 2.17.0 Linux下Bazel构建测试二进制文件失败求助

环境信息 #

• 系统：Ubuntu 20.04 x64
• GCC版本：8.4.0
• Bazel版本：6.5.0
• TensorFlow版本：2.17.0（源码从GitHub下载）

已执行操作 #

1. 下载TensorFlow 2.17.0源码，执行configure完成配置
2. 成功构建核心库：bazel build //tensorflow/core:tensorflow
3. 创建test目录，编写含空main函数的test.cc，并编写BUILD文件依赖核心库
4. 执行bazel build //test:test时出现大量编译错误，直接构建XLA组件bazel build //tensorflow/compiler/tf2xla:xla_compiler --verbose_failures也报相同错误

test.cc 代码 #

int main(int argc, const char** argv) {
    return 0;
}

BUILD 文件内容 #

cc_binary(
  name="test",
  srcs=["test.cc"],
  linkopts = [
        "-lpthread",
        "-lrt",
        "-ldl",
        "-lnsl",
        "-latomic",
  ],
  linkstatic = 1,
  deps=[
    "//tensorflow/core:tensorflow"
  ],
)

.tf_configure.bazelrc 配置 #

build --action_env PYTHON_BIN_PATH="/usr/bin/python3"
build --action_env PYTHON_LIB_PATH="/usr/lib/python3/dist-packages"
build --python_path="/usr/bin/python3"
build:opt --copt=-Wno-sign-compare
build:opt --host_copt=-Wno-sign-compare
test --test_size_filters=small,medium
test:v1 --test_tag_filters=-benchmark-test,-no_oss,-oss_excluded,-gpu,-oss_serial
test:v1 --build_tag_filters=-benchmark-test,-no_oss,-oss_excluded,-gpu
test:v2 --test_tag_filters=-benchmark-test,-no_oss,-oss_excluded,-gpu,-oss_serial,-v1only
test:v2 --build_tag_filters=-benchmark-test,-no_oss,-oss_excluded,-gpu,-v1only

错误日志 #

external/com_google_absl/absl/base/casts.h:161:42: error: no type named 'type' in 'struct std::enable_if<false, int>'
                             int>::type = 0>
                                          ^
In file included from external/local_xla/xla/shape.h:29,
                 from external/local_xla/xla/index_util.h:24,
                 from external/local_xla/xla/literal.h:45,
                 from external/local_xla/xla/literal.cc:16:
external/local_xla/xla/literal.h: In instantiation of 'bool xla::LiteralBase::DeserializeState<InputIterator>::ReadElements(absl::lts_20230802::Span<NativeT>) [with NativeT = ml_dtypes::float8_internal::float8_e4m3fn; InputIterator = const char*]':
external/local_xla/xla/literal.h:1528:44:   required from 'bool xla::LiteralBase::Piece::DeserializeData(xla::LiteralBase::DeserializeState<InputIterator>&) [with NativeT = ml_dtypes::float8_internal::float8_e4m3fn; InputIterator = const char*]':
external/local_xla/xla/literal.h:1587:3:   required from 'xla::Literal::Deserialize(InputIterator, InputIterator) [with InputIterator = const char*]::<lambda(const xla::ShapeIndex&, xla::LiteralBase::Piece*)>::<lambda(auto:5)> [with auto:5 = std::integral_constant<xla::PrimitiveType, (xla::PrimitiveType)20>]':
external/local_xla/xla/primitive_util.h:432:34:   required from 'constexpr R xla::primitive_util::FloatingPointTypeSwitch(F&&, xla::PrimitiveType) [with R = bool; F = xla::Literal::Deserialize(InputIterator, InputIterator) [with InputIterator = const char*]::<lambda(const xla::ShapeIndex&, xla::LiteralBase::Piece*)>::<lambda(auto:5)>]':
external/local_xla/xla/primitive_util.h:480:40:   required from 'constexpr R xla::primitive_util::ArrayTypeSwitch(F&&, xla::PrimitiveType) [with R = bool; F = xla::Literal::Deserialize(InputIterator, InputIterator) [with InputIterator = const char*]::<lambda(const xla::ShapeIndex&, xla::LiteralBase::Piece*)>::<lambda(auto:5)>]':
external/local_xla/xla/literal.h:1587:3:   required from 'static absl::lts_20230802::StatusOr<xla::Literal> xla::Literal::Deserialize(InputIterator, InputIterator) [with InputIterator = const char*]':
external/local_xla/xla/literal.h:1390:62:   required from here
external/local_xla/xla/literal.h:684:64:   in 'constexpr' expansion of 'xla::primitive_util::BitWidth(((xla::PrimitiveType)primitive_type))'
external/local_xla/xla/primitive_util.h:568:54: error: 'constexpr int xla::primitive_util::internal::WidthForType(xla::PrimitiveType) [with const std::array<int, 28>& kWidths = xla::primitive_util::internal::kBitWidths]' called in a constant expression
   return internal::WidthForType<internal::kBitWidths>(type);
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
In file included from external/local_xla/xla/literal.cc:16:
external/local_xla/xla/literal.h:686:23: error: static assertion failed
         static_assert(!primitive_util::IsFloatingPointType(primitive_type));

解决方案 #

这些错误源于GCC 8.4.0对C++17的constexpr特性支持不完善，而TensorFlow 2.17.0的XLA组件依赖了该特性，同时float8类型的处理在旧GCC版本下存在兼容性问题，可尝试以下方法：

1. 升级GCC版本：将GCC升级到9.x及以上版本（推荐10+），Ubuntu 20.04可通过以下命令操作：

   sudo apt install gcc-10 g++-10
   sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100
   sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 100
   

   切换完成后重新执行configure和构建流程。

2. 禁用XLA支持：若暂时无法升级编译器，可在执行configure时选择不启用XLA，跳过相关组件编译以规避错误。

3. 调整Bazel编译选项：在.bazelrc中添加强制C++17标准的配置，确保编译器以正确标准编译：

   build --cxxopt=-std=c++17
   build --host_cxxopt=-std=c++17
   

   注意此方法可能无法完全解决问题，核心兼容性问题仍依赖编译器版本。

内容的提问来源于stack exchange，提问作者WardenAllen