You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

子类化numpy数组:resize时如何重新关联切片子数组

问题

我希望创建一个缓冲数组,扩展numpy的额外索引以直接赋值,替代np.append这类函数,同时让切片得到的“子数组”保持父数组的视图。现有代码已实现基础功能,但还需添加如下特性:当父数组调用self.resize()时,触发回调将父数组重新关联到子数组。

从下方代码可见,子数组的内存地址已改变,但与父数组不一致。我尝试将父数组切片赋值给子数组(child[:] = parent[child.idx:]),却触发错误:ValueError: cannot resize this array: it does not own its data。请问是否有可行的解决办法?

class BufferArray(np.ndarray):
    GROWTH = 2
    def __new__(cls, input_array):
        base = np.asarray(input_array, dtype=float)
        n = base.size
        cap = max(1, 2 * n)                    
        obj = super().__new__(cls, shape=(cap,), dtype=base.dtype)
        obj[:n] = base
        obj[n:] = np.nan
        obj.n = n
        return obj
    
    def __array_finalize__(self, obj):
        if obj is None:
            return
        self.n = getattr(obj, "n", 0)

    def _grow(self):
        old = self.size
        self.resize(old * self.GROWTH, refcheck=False)
        self[old:] = np.nan
        if getattr(self, "mirror_callbacks", False):
            if len(self.mirror_callbacks) > 0:
                for callback in list(self.mirror_callbacks):
                    callback(self)
        
    def update_value(self, value):
        if self.n >= self.size:
            self._grow()
        self[self.n] = value
        self.n += 1
    
    def _attach_mirror(self, mirror):
        print(f"\n_attach_mirror called")
        print(f"parent id: {id(mirror)}")
        print(f"child id before: {id(self)}")
        #self._grow()   # returns ValueError: cannot resize this array: it does not own its data 
        #self[:] = mirror[self.mirror_idx:]  # returns ValueError: cannot resize this array: it does not own its data 
        self = mirror.create_mirror(self.mirror_idx)
        print(f"child id after: {id(self)}")
    
    def _add_mirror_callbacks(self, callback):
        if not getattr(self, "mirror_callbacks", False):
            self.mirror_callbacks=[callback]
        else:
            self.mirror_callbacks.append(callback)
    
    def create_mirror(self, idx):
        slice = self[idx:]
        slice.mirror_idx = idx
        self._add_mirror_callbacks(slice._attach_mirror)
        return slice

parent = BufferArray(np.zeros(2, dtype=int))
child = parent.create_mirror(1)

print(f"parent id: {id(parent)}")
print(f"child id: {id(child)}")
print(f"parent: {parent}")
print(f"child: {child}")

print(f"\nlooping")
for i in range(1, 4):
    parent.update_value(10*i)
    print("\n")
    print(f"parent: {parent}")
    print(f"parent.n: {parent.n}")
    print(f"child: {child}")
    print(f"child.n: {child.n}")

print(f"\nfinal")
print(f"parent: {parent}")
print(f"child: {child}")
解决方案

核心问题是numpy切片生成的视图数组不拥有数据所有权,无法直接resize或重新绑定底层数据。要实现父数组扩容后子数组自动关联新视图,不能通过重新赋值self(方法内的局部变量不会改变外部引用的对象),而是要直接修改子数组的底层属性,让它指向父数组扩容后的新切片。

修改后的代码如下:

import numpy as np

class BufferArray(np.ndarray):
    GROWTH = 2
    def __new__(cls, input_array):
        base = np.asarray(input_array, dtype=float)
        n = base.size
        cap = max(1, 2 * n)                    
        obj = super().__new__(cls, shape=(cap,), dtype=base.dtype)
        obj[:n] = base
        obj[n:] = np.nan
        obj.n = n
        return obj
    
    def __array_finalize__(self, obj):
        if obj is None:
            return
        self.n = getattr(obj, "n", 0)
        self.mirror_idx = getattr(obj, "mirror_idx", 0)
        self.mirror_callbacks = getattr(obj, "mirror_callbacks", [])

    def _grow(self):
        old_size = self.size
        self.resize(old_size * self.GROWTH, refcheck=False)
        self[old_size:] = np.nan
        if hasattr(self, "mirror_callbacks"):
            for callback in list(self.mirror_callbacks):
                callback(self)
        
    def update_value(self, value):
        if self.n >= self.size:
            self._grow()
        self[self.n] = value
        self.n += 1
    
    def _attach_mirror(self, parent):
        # 直接更新当前子数组的底层属性,指向父数组扩容后的新切片
        new_slice = parent[self.mirror_idx:]
        self.base = new_slice.base
        self.shape = new_slice.shape
        self.strides = new_slice.strides
        self.offset = new_slice.offset
        # 同步子数组的有效元素计数
        self.n = max(0, parent.n - self.mirror_idx)
    
    def _add_mirror_callbacks(self, callback):
        if not hasattr(self, "mirror_callbacks"):
            self.mirror_callbacks = []
        self.mirror_callbacks.append(callback)
    
    def create_mirror(self, idx):
        mirror_slice = self[idx:]
        mirror_slice.mirror_idx = idx
        self._add_mirror_callbacks(mirror_slice._attach_mirror)
        return mirror_slice

# 测试代码
parent = BufferArray(np.zeros(2, dtype=int))
child = parent.create_mirror(1)

print(f"parent id: {id(parent)}")
print(f"child id: {id(child)}")
print(f"parent: {parent}")
print(f"child: {child}")

print(f"\nlooping")
for i in range(1, 4):
    parent.update_value(10*i)
    print("\n")
    print(f"parent: {parent}")
    print(f"parent.n: {parent.n}")
    print(f"child: {child}")
    print(f"child.n: {child.n}")

print(f"\nfinal")
print(f"parent: {parent}")
print(f"child: {child}")

关键改动说明:

  1. 在__array_finalize__中统一初始化自定义属性,避免属性缺失导致的错误
  2. _attach_mirror方法不再重新赋值self,而是直接修改子数组的base、shape、strides、offset等底层属性,让它指向父数组扩容后的新切片
  3. 同步子数组的n属性,确保其反映父数组中从镜像索引开始的有效元素数量

这样修改后,父数组扩容时,子数组会自动更新为新的视图,保持与父数组的关联,同时不会触发resize权限错误。

内容的提问来源于stack exchange,提问作者THATS MY QUANT MY QUANTITATIVE

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.06.13 06:08:14