Numpy Structured Array跨平台内存拷贝异常问题求助

问题现象 #

将多个Numpy数组合并为structured arrays后偶发底层数据错误，导致后续C++程序计算异常，目前已用np.where()定位数值差异点：

• Python 3.12.8 + Numpy 2.0.2的MacBook环境下，1000次循环无异常；
• 相同版本环境的Ubuntu Server 24.02上，运行数次迭代后出现数据错误，错误元素数量2-16个不等，触发时机无规律。

环境信息 #

• Python版本：3.12.8（已尝试其他版本）
• Numpy版本：2.0.2（已尝试其他版本）
• 测试设备：
  • MacBook（无异常）
  • Ubuntu Server 24.02（偶发异常）

排查尝试 #

仅调用np.copy()的TestCopy()函数在两台设备上均稳定运行，未出现数据错误。

疑问方向 #

1. 是否为硬件/内存稳定性问题？
2. 是否存在Numpy的OS相关实现差异？
3. 是否为structured array的实现缺陷？

复现代码 #

流程：生成百万级随机Numpy数组 → 传入CoalesceData() → 校验原始数组 → 合并为structured array → 校验数据一致性

import numpy as np

def CoalesceData(arr1, arr2, arr3):
    dtype = [('col1', np.float64), ('col2', np.float64), ('col3', np.float64)]
    structured_arr = np.empty(len(arr1), dtype=dtype)
    structured_arr['col1'] = arr1
    structured_arr['col2'] = arr2
    structured_arr['col3'] = arr3
    return structured_arr

def TestCopy(arr):
    return np.copy(arr)

def RunTest(iterations=1000):
    for i in range(iterations):
        arr1 = np.random.rand(1000000)
        arr2 = np.random.rand(1000000)
        arr3 = np.random.rand(1000000)
        
        # 校验原始数组有效性
        assert not np.isnan(arr1).any(), f"Iter {i}: Original arr1 has NaN"
        assert not np.isnan(arr2).any(), f"Iter {i}: Original arr2 has NaN"
        assert not np.isnan(arr3).any(), f"Iter {i}: Original arr3 has NaN"
        
        # 合并为结构化数组
        structured_arr = CoalesceData(arr1, arr2, arr3)
        
        # 校验数据一致性
        diff_col1 = np.where(structured_arr['col1'] != arr1)[0]
        diff_col2 = np.where(structured_arr['col2'] != arr2)[0]
        diff_col3 = np.where(structured_arr['col3'] != arr3)[0]
        
        if len(diff_col1) > 0 or len(diff_col2) > 0 or len(diff_col3) > 0:
            print(f"Iteration {i} data mismatch detected:")
            print(f"Col1 diff count: {len(diff_col1)}, sample indices: {diff_col1[:10]}...")
            print(f"Col2 diff count: {len(diff_col2)}, sample indices: {diff_col2[:10]}...")
            print(f"Col3 diff count: {len(diff_col3)}, sample indices: {diff_col3[:10]}...")
            return
    print("All iterations completed successfully.")

if __name__ == "__main__":
    RunTest()

输出示例 #

MacBook正常输出 #

All iterations completed successfully.

Ubuntu Server错误输出 #

Iteration 7 data mismatch detected:
Col1 diff count: 5, sample indices: [12345, 67890, 13579, 24680, 98765]...
Col2 diff count: 0, sample indices: []...
Col3 diff count: 3, sample indices: [54321, 87654, 23456]...

内容的提问来源于stack exchange，提问作者Deftness