Numpy数组/DataFrame如何跳过指定区域分片拼接提取子矩阵？

实现方案 #

你可以直接通过提前定义索引掩码的方式跳过显式拼接步骤，比多次调用concatenate效率高很多，也适合批量处理时做内存优化。

1. Numpy数组场景实现

提前固定索引一次定义，循环内直接提取区域累加，无需存储中间结果：

import numpy as np

# 提前定义全局索引，仅需生成1次
row_select_a = np.r_[0:7000, 7201:9800]
row_select_cd = np.r_[7000:7200]
col_select_a = np.r_[0:7000, 7201:9800]
col_select_cd = np.r_[7000:7200]

# 初始化累加容器，可根据数据类型调整dtype节省内存（比如用float32替代默认float64）
sum_a = np.zeros((len(row_select_a), len(col_select_a)), dtype=np.float32)
sum_b = np.zeros((len(row_select_cd), len(col_select_cd)), dtype=np.float32)
sum_c = np.zeros((len(row_select_cd), len(col_select_a)), dtype=np.float32)
sum_d = np.zeros((len(row_select_a), len(col_select_cd)), dtype=np.float32)

# 遍历所有待处理数组
for x in your_array_list:
    sum_a += x[np.ix_(row_select_a, col_select_a)]
    sum_b += x[np.ix_(row_select_cd, col_select_cd)]
    sum_c += x[np.ix_(row_select_cd, col_select_a)]
    sum_d += x[np.ix_(row_select_a, col_select_cd)]

np.ix_用来避免二维索引的广播错误，提取出的区域和你要求的拼接结果完全一致。

2. Pandas DataFrame场景实现

无标签场景用iloc按位置索引即可，逻辑和numpy完全对齐：

import pandas as pd
import numpy as np

# 提前定义全局索引
row_select_a = list(range(0,7000)) + list(range(7201,9800))
row_select_cd = list(range(7000,7200))
col_select_a = list(range(0,7000)) + list(range(7201,9800))
col_select_cd = list(range(7000,7200))

# 初始化累加容器
sum_a = np.zeros((len(row_select_a), len(col_select_a)), dtype=np.float32)
sum_b = np.zeros((len(row_select_cd), len(col_select_cd)), dtype=np.float32)
sum_c = np.zeros((len(row_select_cd), len(col_select_a)), dtype=np.float32)
sum_d = np.zeros((len(row_select_a), len(col_select_cd)), dtype=np.float32)

# 遍历所有待处理DataFrame
for df in your_df_list:
    sum_a += df.iloc[row_select_a, col_select_a].values
    sum_b += df.iloc[row_select_cd, col_select_cd].values
    sum_c += df.iloc[row_select_cd, col_select_a].values
    sum_d += df.iloc[row_select_a, col_select_cd].values

内存优化说明

• 索引仅生成一次，循环内复用，避免重复计算开销
• 直接累加numpy格式的数值，不需要存储每个输入对应的中间a/b/c/d结果，全程仅占用4个输出矩阵的内存
• 可根据你的数值范围选择更小的dtype，比如无浮点需求用int类型，浮点需求优先用float32替代默认float64，内存占用直接减半

内容的提问来源于stack exchange，提问作者Daniel