导入spconv.pytorch遇TensorView编译错误求助

问题 #

导入spconv.pytorch时出现TensorView相关编译错误，核心表现为编译过程中找不到tensorview/pybind_utils.h、tensorview/core/all.h等头文件，最终导致ninja构建失败。

环境信息：

• Python 3.10
• CUDA 11.7
• spconv 2.3.6
• 已尝试安装tensorview 0.4.0、0.2.0版本，重新安装spconv后问题仍未解决

错误输出：

FAILED: /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/tensorview_bind/TensorViewBind.h.gch 
g++ -MMD -MT /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/tensorview_bind/TensorViewBind.h.gch -MF /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/tensorview_bind/TensorViewBind.h.gch.d -I "/usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include" -I "/usr/local/lib/python3.10/dist-packages/pybind11/include" -I "/usr/include/python3.10" -I "/usr/local/cuda/include" -I "/usr/local/lib/python3.10/dist-packages/include" -O3 -std=c++14 -fPIC -x c++-header -DTV_CUDA -c /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/tensorview_bind/TensorViewBind.h -o /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/tensorview_bind/TensorViewBind.h.gch
/usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/tensorview_bind/TensorViewBind.h:1:9: warning: #pragma once in main file
    1 | #pragma once
      |         ^~~~
/usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/tensorview_bind/TensorViewBind.h:2:10: fatal error: tensorview/pybind_utils.h: No such file or directory
    2 | #include <tensorview/pybind_utils.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
[2/39] [GCC][c++/pch]/usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/csrc/arrayref/ArrayPtr.h.gch
FAILED: /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/csrc/arrayref/ArrayPtr.h.gch 
g++ -MMD -MT /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/csrc/arrayref/ArrayPtr.h.gch -MF /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/csrc/arrayref/ArrayPtr.h.gch.d -I "/usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include" -I "/usr/local/lib/python3.10/dist-packages/pybind11/include" -I "/usr/include/python3.10" -I "/usr/local/lib/python3.10/dist-packages/include" -I "/usr/local/cuda/include" -O3 -std=c++14 -fPIC -x c++-header -DTV_CUDA -c /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/csrc/arrayref/ArrayPtr.h -o /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/csrc/arrayref/ArrayPtr.h.gch
/usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/csrc/arrayref/ArrayPtr.h:1:9: warning: #pragma once in main file
    1 | #pragma once
      |         ^~~~
In file included from /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/csrc/arrayref/ArrayPtr.h:3:
/usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/common/TensorViewCPU.h:3:10: fatal error: tensorview/core/all.h: No such file or directory
    3 | #include <tensorview/core/all.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
[3/39] [GCC][c++]/usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/src/tensorview_bind/PyBindTensorViewBind/PyBindTensorViewBind_bind_TensorViewBind.cc.o
FAILED: /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/src/tensorview_bind/PyBindTensorViewBind/PyBindTensorViewBind_bind_TensorViewBind.cc.o 
g++ -MMD -MT /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/src/tensorview_bind/PyBindTensorViewBind/PyBindTensorViewBind_bind_TensorViewBind.cc.o -MF /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/src/tensorview_bind/PyBindTensorViewBind/PyBindTensorViewBind_bind_TensorViewBind.cc.o.d -I "/usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include" -I "/usr/local/lib/python3.10/dist-packages/pybind11/include" -I "/usr/include/python3.10" -I "/usr/local/cuda/include" -I "/usr/local/lib/python3.10/dist-packages/include" -O3 -std=c++14 -fPIC -DTV_CUDA -c /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/src/tensorview_bind/PyBindTensorViewBind/PyBindTensorViewBind_bind_TensorViewBind.cc -o /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/src/tensorview_bind/PyBindTensorViewBind/PyBindTensorViewBind_bind_TensorViewBind.cc.o
In file included from /usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/src/tensorview_bind/PyBindTensorViewBind/PyBindTensorViewBind_bind_TensorViewBind.cc:2:
/usr/local/lib/python3.10/dist-packages/cumm/build/core_cc/include/tensorview_bind/TensorViewBind.h:2:10: fatal error: tensorview/pybind_utils.h: No such file or directory
    2 | #include <tensorview/pybind_utils.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
<ipython-input-39-0c1e6eb5644d> in <cell line: 1>()
----> 1 import spconv.pytorch as spconv

7 frames
/usr/local/lib/python3.10/dist-packages/ccimport/buildtools/writer.py in build_simple_ninja(target, build_dir, sources, build_meta, target_filename, suffix_to_compiler, out_root, verbose, shared, pch_to_sources, pch_to_include, objects_folder, compiler_to_path, linker_to_path, source_meta, linker)
   1107     proc.wait()
   1108     if proc.returncode:
-> 1109         raise subprocess.CalledProcessError(proc.returncode, cmds)
   1110 
   1111     no_work_to_do = False

CalledProcessError: Command '["ninja"]' returned non-zero exit status 1.    

解决方案 #

• 安装版本匹配的tensorview：spconv 2.3.6需搭配tensorview 0.5.0版本执行以下命令安装：

  pip install tensorview==0.5.0
  

• 手动添加头文件路径：若安装后仍找不到头文件，将tensorview的include目录添加到编译环境变量中：

  export C_INCLUDE_PATH=/usr/local/lib/python3.10/dist-packages/tensorview/include:$C_INCLUDE_PATH
  export CPLUS_INCLUDE_PATH=/usr/local/lib/python3.10/dist-packages/tensorview/include:$CPLUS_INCLUDE_PATH
  

• 清理缓存后重新安装spconv：卸载现有相关包，清理缓存后安装对应CUDA版本的预编译包：

  pip uninstall -y spconv cumm tensorview
  rm -rf ~/.cache/pip
  pip install spconv-cu117==2.3.6
  

• 检查gcc版本兼容性：确保gcc版本在7.x~9.x范围内，过高版本会导致C++14编译问题，可通过gcc --version查看，不符则切换到兼容版本。

内容的提问来源于stack exchange，提问作者Sharath