如何将DataFrameGroupBy转换为DataFrame并按每5行计算均值
问题描述
现有数据集需要按每5行计算均值生成新数据表,原始示例数据如下:
,speed_desired,ambient_temperature,ambient_pressure,speed,temperature,pressure 0,1000,19.91,100.98,560.42,148.39,182.19 1,1000,20.03,101.02,843.28,148.37,448.47 2,1000,20.05,100.92,983.12,148.4,675.49 3,1000,20.07,101.0,1052.37,148.24,845.42 4,1000,20.05,100.98,1086.29,148.35,969.79 5,1000,19.93,100.93,1107.29,148.43,1079.25 6,1000,20.06,100.98,1112.75,148.28,1110.86 7,1000,20.0,101.08,1119.72,148.32,1117.46 8,1000,19.94,100.96,1119.33,148.26,1139.64 9,1000,19.94,101.01,1118.8,148.25,1173.16 10,1000,20.05,101.05,1119.16,148.37,1162.44 11,1000,20.0,100.92,1115.65,148.29,1183.07 12,1000,19.97,101.02,1122.59,148.36,1177.44 13,1000,20.1,101.0,1124.66,148.44,1164.61 14,1000,19.99,100.91,1123.38,148.41,1165.45 15,1000,19.96,100.99,1125.05,148.42,1156.23 16,1000,20.06,101.07,1116.51,148.33,1184.12 17,1000,20.1,101.01,1117.51,148.29,1154.25 18,1000,20.03,101.02,1118.27,148.26,1160.29 19,1000,19.92,101.07,1123.47,148.35,1184.8
最初使用如下代码实现分组求均值:
X_data = X_data.groupby(np.arange(len(X_data)) // 5).mean()
该代码可以输出均值计算结果(数值未保留两位小数),输出如下:
0,1000.0,20.0015,100.99666666666667,1102.1871666666666,148.36433333333335,1117.258 1,1000.0,20.003333333333334,100.99033333333334,1120.6831666666667,148.43883333333332,1164.5089999999998 2,1000.0,19.995,100.99233333333333,1120.874,148.54183333333333,1160.0673333333332 3,1000.0,19.988833333333332,100.99600000000001,1119.3431666666665,148.61316666666667,1164.961
分组后对应的标签数据y_data内容如下:
0,M_0003 1,M_0005 2,M_0007 3,M_0003
后续使用处理后的数据执行XGBoost网格搜索训练,代码如下:
variable_params = {'max_depth': [2, 4, 6, 10], 'n_estimators': [5, 10, 20, 25], 'learning_rate': np.linspace(1e-16, 1, 3)} static_params = {'objective': 'multi:softmax', 'num_class': 6, 'silent': 1} bst_grid = GridSearchCV( estimator=XGBClassifier(**static_params), param_grid=variable_params, scoring="accuracy" ) bst_grid.fit(X_data, y_data) print("Best Accuracy:{}".format(bst_grid.best_score_)) for key, value in bst_grid.best_params_.items(): print("{}:{}".format(key, value))
运行时抛出如下错误:
Expected array-like (array or non-string sequence), got <pandas.core.groupby.generic.DataFrameGroupBy object at 0x000001F3B2F17760>
待解决的问题:
- 如何将分组计算后的GroupBy类型数据转换为标准DataFrame?
- 是否存在更合理的按每5行计算均值的实现方案?
解决方案
报错原因很直接:传入模型fit()方法的X_data是GroupBy对象,不是模型要求的类数组结构。正常来说groupby().mean()本身应该返回聚合后的DataFrame,出现这个问题大概率是代码执行顺序异常(比如某步把未聚合的GroupBy对象赋值给了X_data),或者pandas版本差异导致的返回值问题。
两种可直接落地的处理方案:
方案1:修正原有代码,显式生成标准DataFrame
原有分组逻辑本身是可行的,只需要在聚合后做显式处理,顺便统一保留两位小数即可:
import numpy as np # 按每5行分组求均值,保留两位小数,重置索引后输出标准DataFrame X_data = X_data.groupby(np.arange(len(X_data)) // 5).mean().round(2).reset_index(drop=True)
执行完可以用type(X_data)校验,返回pandas.core.frame.DataFrame即为正常。
方案2:分箱打标聚合(鲁棒性更强,避免索引异常)
如果数据集索引不是从0开始的连续整数,原有分组逻辑可能出现分组错位,可以用分箱方法给每行打分组标签再聚合,适配更多场景:
import pandas as pd import numpy as np # 生成每5行一组的分组标签 group_tag = pd.cut( X_data.index, bins=np.arange(0, len(X_data) + 5, 5), right=False, labels=False ) # 按标签聚合求均值 X_data = X_data.groupby(group_tag).mean().round(2).reset_index(drop=True)
训练前务必校验样本数匹配,避免后续报错:
# 输出True代表特征和标签样本数一致,可正常训练 print(len(X_data) == len(y_data))
内容的提问来源于stack exchange,提问作者Patrycja
相关产品推荐
相关产品推荐

