Ubuntu下执行Bazel构建二进制文件时遇Protobuf重复定义错误求助

在Ubuntu 20.04系统上用Bazel完成项目构建后，执行生成的二进制文件时出现Protobuf重复定义错误：

[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/descriptor_database.cc:642] File already exists in database: tensorflow/compiler/mlir/quantization/tensorflow/quantization_options.proto
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/descriptor.cc:1986] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
Aborted (core dumped)

仅依赖TensorFlow（官方仓库commit哈希0db597d0d758aba578783b5bf46c889700a45085），未引入额外插件，仅当依赖@org_tensorflow//tensorflow/compiler/mlir/lite:tf_to_tfl_flatbuffer时触发该问题，移除依赖则无异常。相同代码在Mac OS M1系统上可正常运行。

复现代码 #

WORKSPACE文件

workspace(name = "reproducer")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "bazel_skylib",
    sha256 = "74d544d96f4a5bb630d465ca8bbcfe231e3594e5aae57e1edbf17a6eb3ca2506",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.3.0/bazel-skylib-1.3.0.tar.gz",
        "https://github.com/bazelbuild/bazel-skylib/releases/download/1.3.0/bazel-skylib-1.3.0.tar.gz",
    ],
)

load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")

bazel_skylib_workspace()

local_repository(
    name = "org_tensorflow",
    path = "tensorflow"
)

load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
tf_workspace3()
load("@org_tensorflow//tensorflow:workspace2.bzl", "tf_workspace2")
tf_workspace2()
load("@org_tensorflow//tensorflow:workspace1.bzl", "tf_workspace1")
tf_workspace1()
load("@org_tensorflow//tensorflow:workspace0.bzl", "tf_workspace0")
tf_workspace0()

BUILD文件

load("@org_tensorflow//tensorflow:tensorflow.bzl", "tf_cc_binary")

tf_cc_binary(
    name = "test",
    srcs = ["test.cc"],
    deps = [
        "@org_tensorflow//tensorflow/compiler/mlir/lite:tf_to_tfl_flatbuffer",
    ],
)

test.cc文件

#include <iostream>

int main(int argc, char *argv[]) {
    std::cout << "Hello World!" << std::endl;

    return 0;
}

该问题源于TensorFlow的MLIR Lite模块中quantization_options.proto被多次链接进二进制文件，导致Protobuf描述符冲突，针对Ubuntu 20.04环境可尝试以下修复方式：

1. 强制静态链接 #

修改BUILD文件中的tf_cc_binary规则，添加linkstatic = True参数，确保所有依赖以静态方式链接，避免重复的Protobuf描述符被多次加载：

tf_cc_binary(
    name = "test",
    srcs = ["test.cc"],
    deps = [
        "@org_tensorflow//tensorflow/compiler/mlir/lite:tf_to_tfl_flatbuffer",
    ],
    linkstatic = True,
)

2. 统一Protobuf依赖版本 #

在WORKSPACE文件中，先显式指定与TensorFlow兼容的Protobuf版本，再加载TensorFlow的workspace规则，避免依赖版本冲突：

# 先添加与TensorFlow兼容的Protobuf版本（建议查看对应commit的TensorFlow WORKSPACE文件确认版本）
http_archive(
    name = "com_google_protobuf",
    sha256 = "6a6b65e9d0b5c54c6e4d8731804b3df03e088d1a4e3d3d772e2e3d6a7e3c3e0d",
    strip_prefix = "protobuf-3.20.3",
    urls = [
        "https://github.com/protocolbuffers/protobuf/releases/download/v3.20.3/protobuf-all-3.20.3.tar.gz",
    ],
)

# 再加载TensorFlow的workspace规则
load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
tf_workspace3()
load("@org_tensorflow//tensorflow:workspace2.bzl", "tf_workspace2")
tf_workspace2()
load("@org_tensorflow//tensorflow:workspace1.bzl", "tf_workspace1")
tf_workspace1()
load("@org_tensorflow//tensorflow:workspace0.bzl", "tf_workspace0")
tf_workspace0()

3. 临时绕过描述符检查 #

如果上述方法无效，可通过设置环境变量绕过Protobuf的重复描述符检查（仅作为临时方案，不建议长期使用）：

PROTOBUF_ALLOW_DUPLICATE_PROTOS=1 ./bazel-bin/test

4. 升级Bazel版本 #

Ubuntu 20.04默认的Bazel版本可能较旧，尝试升级到TensorFlow推荐的版本（对应commit 0db597d建议使用Bazel 5.3.0及以上），旧版本链接器的处理逻辑可能导致重复符号问题。

内容的提问来源于stack exchange，提问作者mactok