Cython中array赋值memoryview为何存在Python交互及优化判断

问题描述 #

使用cython -a my-file.pyx编译时，以下代码中的cdef行在生成的HTML报告中被标记为黄色：

# my-file.pyx
from cpython.array cimport array

def f(double[:] xyz):
    cdef double[:] inv2 = array('d', [xyz[0]*3, xyz[1], xyz[2]*3])

原本预期该行无Python交互，请问这个标记合理吗？另外，除了HTML的颜色标记，如何判断代码是否存在Python交互？黄色标记时怎么判断有没有优化空间？

对应的C代码如下：

__pyx_t_1 = 0;
  __pyx_t_2 = -1;
  if (__pyx_t_1 < 0) {
    __pyx_t_1 += __pyx_v_xyz.shape[0];
    if (unlikely(__pyx_t_1 < 0)) __pyx_t_2 = 0;
  } else if (unlikely(__pyx_t_1 >= __pyx_v_xyz.shape[0])) __pyx_t_2 = 0;
  if (unlikely(__pyx_t_2 != -1)) {
    __Pyx_RaiseBufferIndexError(__pyx_t_2);
    __PYX_ERR(0, 5, __pyx_L1_error)
  }
  __pyx_t_3 = PyFloat_FromDouble(((*((double *) ( /* dim=0 */ (__pyx_v_xyz.data + __pyx_t_1 * __pyx_v_xyz.strides[0]) ))) * 3.0)); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_3);
  __pyx_t_1 = 1;
  __pyx_t_2 = -1;
  if (__pyx_t_1 < 0) {
    __pyx_t_1 += __pyx_v_xyz.shape[0];
    if (unlikely(__pyx_t_1 < 0)) __pyx_t_2 = 0;
  } else if (unlikely(__pyx_t_1 >= __pyx_v_xyz.shape[0])) __pyx_t_2 = 0;
  if (unlikely(__pyx_t_2 != -1)) {
    __Pyx_RaiseBufferIndexError(__pyx_t_2);
    __PYX_ERR(0, 5, __pyx_L1_error)
  }
  __pyx_t_4 = PyFloat_FromDouble((*((double *) ( /* dim=0 */ (__pyx_v_xyz.data + __pyx_t_1 * __pyx_v_xyz.strides[0]) )))); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_4);
  __pyx_t_1 = 2;
  __pyx_t_2 = -1;
  if (__pyx_t_1 < 0) {
    __pyx_t_1 += __pyx_v_xyz.shape[0];
    if (unlikely(__pyx_t_1 < 0)) __pyx_t_2 = 0;
  } else if (unlikely(__pyx_t_1 >= __pyx_v_xyz.shape[0])) __pyx_t_2 = 0;
  if (unlikely(__pyx_t_2 != -1)) {
    __Pyx_RaiseBufferIndexError(__pyx_t_2);
    __PYX_ERR(0, 5, __pyx_L1_error)
  }
  __pyx_t_5 = PyFloat_FromDouble(((*((double *) ( /* dim=0 */ (__pyx_v_xyz.data + __pyx_t_1 * __pyx_v_xyz.strides[0]) ))) * 3.0)); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_5);
  __pyx_t_6 = PyList_New(3); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_6);
  __Pyx_GIVEREF(__pyx_t_3);
  PyList_SET_ITEM(__pyx_t_6, 0, __pyx_t_3);
  __Pyx_GIVEREF(__pyx_t_4);
  PyList_SET_ITEM(__pyx_t_6, 1, __pyx_t_4);
  __Pyx_GIVEREF(__pyx_t_5);
  PyList_SET_ITEM(__pyx_t_6, 2, __pyx_t_5);
  __pyx_t_3 = 0;
  __pyx_t_4 = 0;
  __pyx_t_5 = 0;
  __pyx_t_5 = PyTuple_New(2); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_5);
  __Pyx_INCREF(__pyx_n_s_d);
  __Pyx_GIVEREF(__pyx_n_s_d);
  PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_n_s_d);
  __Pyx_GIVEREF(__pyx_t_6);
  PyTuple_SET_ITEM(__pyx_t_5, 1, __pyx_t_6);
  __pyx_t_6 = 0;
  __pyx_t_6 = __Pyx_PyObject_Call(((PyObject *)__pyx_ptype_7cpython_5array_array), __pyx_t_5, NULL); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_6);
  __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0;
  __pyx_t_7 = __Pyx_PyObject_to_MemoryviewSlice_ds_double(__pyx_t_6, PyBUF_WRITABLE); if (unlikely(!__pyx_t_7.memview)) __PYX_ERR(0, 5, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
  __pyx_v_inv2 = __pyx_t_7;
  __pyx_t_7.memview = NULL;
  __pyx_t_7.data = NULL;

回答 #

1. 黄色标记完全合理 #

黄色标记说明该行存在Python层面的交互操作，从生成的C代码能清晰看到原因：

• [xyz[0]*3, xyz[1], xyz[2]*3]列表字面量会触发Python列表创建（对应PyList_New(3)），且每个元素都会通过PyFloat_FromDouble把C级别的double转换成Python float对象。
• 调用array('d', ...)本质是调用Python的array构造函数，对应__Pyx_PyObject_Call，属于纯Python层面的调用。
• 虽然最后把array转换成了Cython的memoryview，但前面的步骤已经包含大量Python API调用，所以Cython会把整行标记为黄色。

2. 判断Python交互的其他方法 #

除了HTML颜色标记，直接分析生成的C代码即可：

• 查找是否存在Python C API函数调用，比如PyFloat_FromDouble、PyList_New、PyObject_Call这类前缀为Py或__Pyx_Py的函数，出现这些就说明存在Python交互。
• 检查是否有Python对象（如列表、float实例）的创建操作，这类操作必然涉及Python runtime调用。

3. 黄色标记时的优化空间判断 #

针对生成的C代码，找出可替换为纯C级操作的部分：

• 当前代码中，创建Python列表、C double转Python float、调用Python array构造函数都是核心优化点。
• 优化方案：用纯C级别的内存操作或Cython原生数组替代Python array构造，示例如下：

  def f(double[:] xyz):
      cdef double[:] inv2 = double[3]()
      inv2[0] = xyz[0] * 3
      inv2[1] = xyz[1]
      inv2[2] = xyz[2] * 3
  

  改写后整行会变成白色（无Python交互），因为所有操作都是纯C级别的内存访问和数值计算，完全绕开了Python对象创建和API调用。

内容的提问来源于stack exchange，提问作者nos