Mac平台Intel芯片下Numpy内存占用异常问题问询

我生成了一个尺寸为(720, 720, 3)的NumPy数组，理论内存占用应为720 * 720 * 3 * 8Byte = 12.3MB。但调用ans = memory_benchmark()后，实际内存占用高达188MB，远高于预期——我认为它应该和m1 = np.ones((720, 720, 3))的内存占用一致。

测试环境 #

• 环境1：numpy=1.23.4, memory_profiler=0.61.0, python=3.10.6, MacOS 12.6.1（非M1 Intel芯片）
• 环境2：numpy=1.19.5, memory_profiler=0.61.0, python=3.8.15, MacOS 12.6.1（非M1 Intel芯片）

内存分析代码 #

import numpy as np
from memory_profiler import profile


@profile
def memory_benchmark():
    m1 = np.ones((720, 720, 3))
    m2 = np.random.randint(128, size=(720, 720, 77, 3))
    a = m2[:, :, :, 0].astype(np.uint16)
    b = m2[:, :, :, 1].astype(np.uint16)
    ans = np.array(m1[b, a].sum(axis=2))
    m2 = None
    a = None
    b = None
    m1 = None
    return ans


@profile
def f():
    ans = memory_benchmark()
    print(ans.shape)
    print("finished")


if __name__ == '__main__':
    f()

内存分析结果 #

(720, 720, 3)
finished

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     5     59.3 MiB     59.3 MiB           1   @profile
     6                                         def memory_benchmark():
     7     71.2 MiB     11.9 MiB           1       m1 = np.ones((720, 720, 3))
     8    984.8 MiB    913.7 MiB           1       m2 = np.random.randint(128, size=(720, 720, 77, 3))
     9   1061.0 MiB     76.1 MiB           1       a = m2[:, :, :, 0].astype(np.uint16)
    10   1137.1 MiB     76.1 MiB           1       b = m2[:, :, :, 1].astype(np.uint16)
    11   1160.9 MiB     23.8 MiB           1       ans = np.array(m1[b, a].sum(axis=2))
    12    247.3 MiB   -913.6 MiB           1       m2 = None
    13    247.3 MiB      0.0 MiB           1       a = None
    14    247.3 MiB      0.0 MiB           1       b = None
    15    247.3 MiB      0.0 MiB           1       m1 = None
    16    247.3 MiB      0.0 MiB           1       return ans


Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
    19     59.3 MiB     59.3 MiB           1   @profile
    20                                         def f():
    21    247.3 MiB    188.0 MiB           1       ans = memory_benchmark()
    22    247.3 MiB      0.0 MiB           1       print(ans.shape)
    23    247.3 MiB      0.0 MiB           1       print("finished")

类型信息 #

• print(type(m1[0, 0, 0]))输出：<class 'numpy.float64'>
• print(type(m2[0, 0, 0, 0]))输出：<class 'numpy.int64'>
• print(type(ans[0, 0, 0]))输出：<class 'numpy.float64'>

该问题仅出现在MacOS环境中，Ubuntu虚拟机下无此异常。

内容的提问来源于stack exchange，提问作者maplemaple