终端运行Python groupby transform仅返回NaN且结果行数异常问题
Pandas transform返回NaN且结果异常(终端运行异常但Jupyter正常)
在终端运行以下代码为DataFrame新增列时,得到全NaN值,且结果行数不符合预期,但相同代码在Jupyter中可正常执行,直接给列赋值常量(如100)能正常生效:
df_null_count_extract['DAY_AVG'] = df_null_count_extract.groupby(['CONTROL_COL_NAME', 'RUN_ID', 'RUN_DATE', 'CONTROL_TABLE_NAME'], as_index = False)['CONTROL_COL_VAL'].transform('mean')
解决方向:
移除groupby的
as_index=False参数
transform方法不需要该参数,它会自动返回与原DataFrame同索引的结果,旧版本pandas中设置as_index=False可能导致输出结构异常。修改后代码:df_null_count_extract['DAY_AVG'] = df_null_count_extract.groupby(['CONTROL_COL_NAME', 'RUN_ID', 'RUN_DATE', 'CONTROL_TABLE_NAME'])['CONTROL_COL_VAL'].transform('mean')检查分组列的数据类型一致性
终端与Jupyter环境中,分组列(如RUN_DATE)可能存在类型差异(比如Jupyter中是datetime类型,终端中是字符串),导致分组键不匹配,每个组无有效数据从而计算出NaN。先打印列类型排查:print(df_null_count_extract[['CONTROL_COL_NAME', 'RUN_ID', 'RUN_DATE', 'CONTROL_TABLE_NAME']].dtypes)若存在类型不一致,统一转换,例如将日期列转为datetime:
df_null_count_extract['RUN_DATE'] = pd.to_datetime(df_null_count_extract['RUN_DATE'])重置原DataFrame索引
若原DataFrame存在重复索引或索引混乱,transform返回的结果无法正确对齐到原数据。重置索引后再执行分组计算:df_null_count_extract = df_null_count_extract.reset_index(drop=True) df_null_count_extract['DAY_AVG'] = df_null_count_extract.groupby(['CONTROL_COL_NAME', 'RUN_ID', 'RUN_DATE', 'CONTROL_TABLE_NAME'])['CONTROL_COL_VAL'].transform('mean')改用分组计算+merge的替代方案
如果transform仍有问题,用更稳妥的方式实现:先计算分组均值,再合并回原DataFrame:# 计算每个分组的均值 grouped_mean = df_null_count_extract.groupby(['CONTROL_COL_NAME', 'RUN_ID', 'RUN_DATE', 'CONTROL_TABLE_NAME'], as_index=False)['CONTROL_COL_VAL'].mean().rename(columns={'CONTROL_COL_VAL': 'DAY_AVG'}) # 合并到原表 df_null_count_extract = pd.merge(df_null_count_extract, grouped_mean, on=['CONTROL_COL_NAME', 'RUN_ID', 'RUN_DATE', 'CONTROL_TABLE_NAME'], how='left')
内容的提问来源于stack exchange,提问作者ArchAngelPwn
相关产品推荐
相关产品推荐

