MKL与OpenBLAS链接交互问题:动态库符号为何覆盖静态链接库符号的咨询
MKL与OpenBLAS链接交互问题:动态库符号为何覆盖静态链接库符号的咨询
我现在遇到一个关于动态/静态链接符号优先级的棘手问题,想请教各位大佬:
我正在使用的R程序是动态链接到通用BLAS库的(大部分情况下是OpenBLAS)。现在我在R内部通过dlopen()动态加载了另一个共享库libtorch.so,而这个libtorch是静态链接到MKL BLAS的。
按照我对静态和动态链接的理解,这种场景应该不会有问题——毕竟libtorch已经把MKL静态链接进去了,调用libtorch的代码时,应该优先使用它自己内部的符号,不会去调用其他动态加载的同名符号才对。
我之前做过类似的验证测试:编译一个可执行文件,同时动态链接到实现了print()函数的libA,以及静态链接了libA的libB。调用libB里的代码时,确实会正确调用它自己内部版本的libA中的print(),完全不受外部动态链接的libA影响。
但到了libtorch/MKL和OpenBLAS的组合上,情况就不对了:如果我编译一个同时动态链接libTorch和OpenBLAS的可执行文件,libtorch居然会调用OpenBLAS的例程,而不是它自己静态链接的MKL版本。
比如下面的调用栈清晰显示sgemm_来自OpenBLAS:
#0 0x00007ffff5537da0 in sgemm_ () from /lib/x86_64-linux-gnu/libopenblas.so.0 #1 0x00007fffde5385d6 in at::native::cpublas::gemm(at::native::TransposeType, at::native::TransposeType, long, long, long, float, float const*, long, float const*, long, float, float*, long) () from /home/rstudio/data/torch/build-lantern/libtorch/lib/libtorch_cpu.so #2 0x00007fffde67c139 in at::native::addmm_impl_cpu_(at::Tensor&, at::Tensor const&, at::Tensor, at::Tensor, c10::Scalar const&, c10::Scalar const&) () from /home/rstudio/data/torch/build-lantern/libtorch/lib/libtorch_cpu.so #3 0x00007fffde67d475 in at::native::structured_mm_out_cpu::impl(at::Tensor const&, at::Tensor const&, at::Tensor const&) () from /home/rstudio/data/torch/build-lantern/libtorch/lib/libtorch_cpu.so #4 0x00007fffdf42309b in at::(anonymous namespace)::wrapper_CPU_mm(at::Tensor const&, at::Tensor const&) () from /home/rstudio/data/torch/build-lantern/libtorch/lib/libtorch_cpu.so #5 0x00007fffdf423123 in c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, at::Tensor const&), &at::(anonymous namespace)::wrapper_CPU_mm>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, at::Tensor const&> >, at::Tensor (at::Tensor const&, at::Tensor const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&) () from /home/rstudio/data/torch/build-lantern/libtorch/lib/libtorch_cpu.so #6 0x00007fffdf1eaa70 in at::_ops::mm::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&) () from /home/rstudio/data/torch/build-lantern/libtorch/lib/libtorch_cpu.so
更奇怪的是,libtorch_cpu.so里明明有自己的sgemm_符号:
nm libtorch/lib/libtorch_cpu.so | grep "T sgemm_" 0000000006c531b0 T sgemm_ 0000000006c53870 T sgemm_64 0000000006c53870 T sgemm_64_
我实在搞不懂,什么情况下动态加载的库的符号会覆盖掉静态链接进另一个库的符号?肯定是我漏掉了链接机制里的某个关键知识点,希望能得到大家的指点和建议!
可重现示例
C++代码
#include <torch/torch.h> #include <iostream> #include <cblas.h> extern "C" void execute () { for (auto i = 1; i < 10; i++) { torch::Tensor tensor = torch::randn({2000, 2000}); auto k = tensor.mm(tensor); } } int main() { int m = 3; // rows of A int n = 3; // cols of A // Matrix A (m x n) in row-major order double A[] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0}; // Vector x (size n) double x[] = {1.0, 1.0, 1.0}; // Result vector y (size m), initially zero double y[] = {0.0, 0.0, 0.0}; // Scalar multipliers double alpha = 1.0, beta = 0.0; // Perform y = alpha * A * x + beta * y cblas_dgemv(CblasRowMajor, CblasNoTrans, m, n, alpha, A, n, x, 1, beta, y, 1); execute(); return 0; }
CMakeLists.txt
set(CMAKE_POSITION_INDEPENDENT_CODE ON) cmake_minimum_required(VERSION 3.0 FATAL_ERROR) project(example) find_package(Torch REQUIRED) find_package(BLAS) add_executable(example example.cpp) target_link_libraries(example "${TORCH_LIBRARIES}" "${BLAS_LIBRARIES}") set_property(TARGET example PROPERTY CXX_STANDARD 17)
编译步骤
- 下载对应版本的LibTorch(CPU版)
- 执行以下编译命令:
mkdir build && cd build cmake .. -DCMAKE_PREFIX_PATH=<path to libtorch> cmake --build .
备注:内容来源于stack exchange,提问作者Daniel Falbel
相关产品推荐
相关产品推荐

