Cython并行调用基于FFTW的SSHT外部C库时出现内存释放及FFTW断言错误
我此前使用Python的multiprocess库并行实现了一段计算代码,可正常运行但速度较慢、内存占用较高,因此决定尝试将其改写为Cython代码。我对Cython、C语言经验较少,下面的示例已尽可能简化,该代码在串行模式下可正常运行,但一旦开启并行就无法工作。由于并行运行的特性,我梳理了全部代码并关闭了GIL。
该代码依赖外部C库SSHT(球谐变换工具库),编译说明见对应项目的README文档,底层调用了FFTW。SSHT仓库自带的Cython封装中调用了我正在使用的同一个C函数ssht_core_mw_inverse_sov_sym_ss,与我的实现逻辑最接近的仓库内置Cython代码如下:
def ssht_inverse_mwss_complex( np.ndarray[ double complex, ndim=1, mode="c"] f_lm not None, int L, int spin ): cdef ssht_dl_method_t dl_method = SSHT_DL_RISBO f_mwss_c = np.empty([L+1,2*L,], dtype=complex) ssht_core_mw_inverse_sov_sym_ss( <double complex*> np.PyArray_DATA(f_mwss_c), <const double complex*> np.PyArray_DATA(f_lm), L, spin, dl_method, 0 ) return f_mwss_c
由于我需要无GIL运行的版本,因此不得不本地重新实现了该封装逻辑。
使用Cython模块运行脚本时会触发段错误,每次报错略有不同:要么没有明确报错信息,要么提示指针内存分配相关错误:
malloc: *** error for object 0x7f83de8b58e0: pointer being freed was not allocated python(21855,0x70000d333000) malloc: Double free of object 0x7f83de8b58e0 python(21855,0x70000d536000) malloc: *** set a breakpoint in malloc_error_break to debug
要么报错和FFTW相关:
fftw: /Users/runner/.conan/data/fftw/3.3.8/_/_/build/55f3919d9a41efc78a625ee65e5d1ea60d02b2ff/source_subfolder/kernel/planner.c:261: assertion failed: SLVNDX(slot) == slvndx
我查阅技术资料时发现有同类问题的反馈,因此担心是FFTW的设计特性导致我无法实现目标(毕竟我C语言经验不足)。
我尝试调用libc.stdlib提供的free接口,但没有解决问题;也尝试使用cython.view创建数组,但无法生成SSHT库要求的double complex类型;还尝试配置Cython调试模式,但在我的Mac上运行遇到了问题。我已经为此耗费了两天时间。
我使用常规命令python setup.py build_ext --inplace编译扩展,使用python3.8.5、Cython==0.29.21,运行在macOS 11.0.1系统上。
我的Cython文件如下:
import numpy as np from libc.stdio cimport printf from libc.stdlib cimport calloc, malloc from cython.parallel import parallel, prange from openmp cimport omp_get_thread_num # needed to recreate without importing (for nogil) cdef extern from "ssht/ssht.h" nogil: ctypedef enum ssht_dl_method_t: SSHT_DL_RISBO, SSHT_DL_TRAPANI void ssht_core_mw_inverse_sov_sym_ss( double complex *f, const double complex *flm, int L, int spin, ssht_dl_method_t dl_method, int verbosity ) def my_cython_module(int L, int threads): """ dummy function more to show that parallel loops fails """ cdef int ell, tid with nogil, parallel(num_threads=threads): tid = omp_get_thread_num() for ell in prange(L * L, schedule="guided"): printf("ell: %i\n", ell) _ssht_inverse(L, ell) cdef double complex * _ssht_inverse(int L, int ind) nogil: """ function creates a 1D complex array flm with zeros and a 1 then calls c function to get 2D complex array f not returning anything as it's just for demonstration """ cdef ssht_dl_method_t dl_method = SSHT_DL_RISBO cdef double complex *flm = NULL cdef double complex *f = NULL flm = <double complex *> calloc(L * L, sizeof(double complex)) flm[ind] = 1 f = <double complex *> malloc((L + 1) * (2 * L) * sizeof(double complex)) ssht_core_mw_inverse_sov_sym_ss(f, flm, L, 0, dl_method, 0) return f
我的setup.py文件如下:
import os from Cython.Build import cythonize from setuptools import Extension, setup # running on mac so need GCC instead of clang os.environ["CC"] = "gcc-10" setup( ext_modules=cythonize( Extension( "test", ["*.pyx"], extra_compile_args=["-fopenmp"], extra_link_args=["-fopenmp"], include_dirs=["/usr/local/include"], ), annotate=True, language_level=3, compiler_directives=dict(boundscheck=False, embedsignature=True), ), )
以下示例(可通过pip install pyssht安装依赖)可正常并行运行,因此判断问题出在C/Cython实现环节:
# the cython wrapper from the external library from pyssht import ssht_inverse_mwss_complex import numpy as np from multiprocess import Pool def my_python_implementation(L, threads): """ the python equivalent in parallel """ def func(chunk): """ deals with each chunk """ for ell in chunk: print(f"ell: {ell}") flm = np.zeros(L * L, dtype=np.complex_) flm[ell] = 1 ssht_inverse_mwss_complex(flm, L, 0) chunks = np.array_split(np.arange(L * L), threads) with Pool(processes=threads) as p: p.map(func, chunks)
既然能通过Python实现并行运行,非常希望能找到Cython并行方案的可行实现路径。
内容的提问来源于Stack Exchange,提问作者Paddy Roddy

