子类化numpy数组:resize时如何重新关联切片子数组
问题
我希望创建一个缓冲数组,扩展numpy的额外索引以直接赋值,替代np.append这类函数,同时让切片得到的“子数组”保持父数组的视图。现有代码已实现基础功能,但还需添加如下特性:当父数组调用self.resize()时,触发回调将父数组重新关联到子数组。
从下方代码可见,子数组的内存地址已改变,但与父数组不一致。我尝试将父数组切片赋值给子数组(child[:] = parent[child.idx:]),却触发错误:ValueError: cannot resize this array: it does not own its data。请问是否有可行的解决办法?
class BufferArray(np.ndarray): GROWTH = 2 def __new__(cls, input_array): base = np.asarray(input_array, dtype=float) n = base.size cap = max(1, 2 * n) obj = super().__new__(cls, shape=(cap,), dtype=base.dtype) obj[:n] = base obj[n:] = np.nan obj.n = n return obj def __array_finalize__(self, obj): if obj is None: return self.n = getattr(obj, "n", 0) def _grow(self): old = self.size self.resize(old * self.GROWTH, refcheck=False) self[old:] = np.nan if getattr(self, "mirror_callbacks", False): if len(self.mirror_callbacks) > 0: for callback in list(self.mirror_callbacks): callback(self) def update_value(self, value): if self.n >= self.size: self._grow() self[self.n] = value self.n += 1 def _attach_mirror(self, mirror): print(f"\n_attach_mirror called") print(f"parent id: {id(mirror)}") print(f"child id before: {id(self)}") #self._grow() # returns ValueError: cannot resize this array: it does not own its data #self[:] = mirror[self.mirror_idx:] # returns ValueError: cannot resize this array: it does not own its data self = mirror.create_mirror(self.mirror_idx) print(f"child id after: {id(self)}") def _add_mirror_callbacks(self, callback): if not getattr(self, "mirror_callbacks", False): self.mirror_callbacks=[callback] else: self.mirror_callbacks.append(callback) def create_mirror(self, idx): slice = self[idx:] slice.mirror_idx = idx self._add_mirror_callbacks(slice._attach_mirror) return slice parent = BufferArray(np.zeros(2, dtype=int)) child = parent.create_mirror(1) print(f"parent id: {id(parent)}") print(f"child id: {id(child)}") print(f"parent: {parent}") print(f"child: {child}") print(f"\nlooping") for i in range(1, 4): parent.update_value(10*i) print("\n") print(f"parent: {parent}") print(f"parent.n: {parent.n}") print(f"child: {child}") print(f"child.n: {child.n}") print(f"\nfinal") print(f"parent: {parent}") print(f"child: {child}")
解决方案
核心问题是numpy切片生成的视图数组不拥有数据所有权,无法直接resize或重新绑定底层数据。要实现父数组扩容后子数组自动关联新视图,不能通过重新赋值self(方法内的局部变量不会改变外部引用的对象),而是要直接修改子数组的底层属性,让它指向父数组扩容后的新切片。
修改后的代码如下:
import numpy as np class BufferArray(np.ndarray): GROWTH = 2 def __new__(cls, input_array): base = np.asarray(input_array, dtype=float) n = base.size cap = max(1, 2 * n) obj = super().__new__(cls, shape=(cap,), dtype=base.dtype) obj[:n] = base obj[n:] = np.nan obj.n = n return obj def __array_finalize__(self, obj): if obj is None: return self.n = getattr(obj, "n", 0) self.mirror_idx = getattr(obj, "mirror_idx", 0) self.mirror_callbacks = getattr(obj, "mirror_callbacks", []) def _grow(self): old_size = self.size self.resize(old_size * self.GROWTH, refcheck=False) self[old_size:] = np.nan if hasattr(self, "mirror_callbacks"): for callback in list(self.mirror_callbacks): callback(self) def update_value(self, value): if self.n >= self.size: self._grow() self[self.n] = value self.n += 1 def _attach_mirror(self, parent): # 直接更新当前子数组的底层属性,指向父数组扩容后的新切片 new_slice = parent[self.mirror_idx:] self.base = new_slice.base self.shape = new_slice.shape self.strides = new_slice.strides self.offset = new_slice.offset # 同步子数组的有效元素计数 self.n = max(0, parent.n - self.mirror_idx) def _add_mirror_callbacks(self, callback): if not hasattr(self, "mirror_callbacks"): self.mirror_callbacks = [] self.mirror_callbacks.append(callback) def create_mirror(self, idx): mirror_slice = self[idx:] mirror_slice.mirror_idx = idx self._add_mirror_callbacks(mirror_slice._attach_mirror) return mirror_slice # 测试代码 parent = BufferArray(np.zeros(2, dtype=int)) child = parent.create_mirror(1) print(f"parent id: {id(parent)}") print(f"child id: {id(child)}") print(f"parent: {parent}") print(f"child: {child}") print(f"\nlooping") for i in range(1, 4): parent.update_value(10*i) print("\n") print(f"parent: {parent}") print(f"parent.n: {parent.n}") print(f"child: {child}") print(f"child.n: {child.n}") print(f"\nfinal") print(f"parent: {parent}") print(f"child: {child}")
关键改动说明:
- 在
__array_finalize__中统一初始化自定义属性,避免属性缺失导致的错误 _attach_mirror方法不再重新赋值self,而是直接修改子数组的base、shape、strides、offset等底层属性,让它指向父数组扩容后的新切片- 同步子数组的
n属性,确保其反映父数组中从镜像索引开始的有效元素数量
这样修改后,父数组扩容时,子数组会自动更新为新的视图,保持与父数组的关联,同时不会触发resize权限错误。
内容的提问来源于stack exchange,提问作者THATS MY QUANT MY QUANTITATIVE
相关产品推荐
相关产品推荐

