IndexError报错求助:数组维度不匹配,前两次循环正常后续失败
问题解决:IndexError与分组异常值处理
错误根源
- 浮点数精度陷阱:代码中用
row[0] == n做匹配,但n = n + 0.1会产生浮点数精度误差(比如0.3无法用二进制精确表示),导致第三次循环时没有匹配到任何数据,tmp变成空数组。空数组是一维的,执行tmp[:,2]时就会触发"too many indices for array"错误。 - 排序逻辑错误:
np.sort(d_us)会对每个子数组的元素单独排序,而非按第0列对整个数组排序,导致分组的原始数据顺序混乱。 - 未处理空分组:没有判断
tmp是否为空就直接进行索引和统计计算,空数组会直接引发后续报错。
修复后的代码(推荐用Pandas,更简洁可靠)
Pandas的分组功能能完美解决浮点数匹配和空分组问题,同时简化异常值计算:
import pandas as pd import datetime # 读取数据 df = pd.read_csv("test.csv", header=None, names=["col0", "col1"]) # 按第0列分组(自动处理浮点数精度问题) grouped = df.groupby("col0") # 输出文件时间戳 nts = datetime.datetime.now().timestamp() output_path = f"calib_temp-{nts}.csv" # 遍历每个分组,计算并导出异常值 with open(output_path, 'w', newline='') as f: writer = pd.io.parsers.csv.writer(f) for _, group in grouped: # 计算差值列 group["diff"] = round(group["col0"] - group["col1"], 1) # 计算均值和标准差 mean_diff = group["diff"].mean() std_diff = group["diff"].std() # 跳过只有一个数据的分组(无标准差) if std_diff == 0: continue # 计算z-score,筛选异常值 group["z_score"] = (group["diff"] - mean_diff) / std_diff outliers = group[abs(group["z_score"]) > 1] # 写入异常值 writer.writerows(outliers[["col0", "col1"]].values.tolist())
若坚持使用Numpy的修复方案
需要处理浮点数精度和空分组判断:
import numpy as np import csv import datetime # 读取数据 d_us = [] with open("test.csv", 'r', newline='') as drh: fr_rh = csv.reader(drh, delimiter=',') for row in fr_rh: d_us.append([float(row[0]), float(row[1])]) # 按第0列排序(正确的排序方式) d = np.array(d_us) d = d[np.argsort(d[:, 0])] # 输出文件时间戳 nts = datetime.datetime.now().timestamp() output_path = f"calib_temp-{nts}.csv" # 获取所有唯一的第0列值(去重,避免浮点数循环的精度问题) unique_col0 = np.unique(d[:, 0]) for n in unique_col0: # 用isclose处理浮点数精度匹配 mask = np.isclose(d[:, 0], n) tmp = d[mask] if len(tmp) == 0: continue # 计算差值列 diff = np.round(tmp[:, 0] - tmp[:, 1], 1) tmp_with_diff = np.column_stack((tmp, diff)) # 计算统计值 mean_diff = np.mean(tmp_with_diff[:, 2]) std_diff = np.std(tmp_with_diff[:, 2]) if std_diff == 0: continue # 筛选异常值并写入 z_scores = (tmp_with_diff[:, 2] - mean_diff) / std_diff outliers = tmp_with_diff[np.abs(z_scores) > 1] with open(output_path, 'a', newline='') as ct: c = csv.writer(ct, delimiter=',') c.writerows(outliers[:, :2].tolist())
内容的提问来源于stack exchange,提问作者Binsky734
相关产品推荐
相关产品推荐

