为何Pandas过滤后输出为NaN？求实现非NaN输出的解决方法

问题现象 #

执行以下代码行时，无法获取有效数值，返回NaN：

f_bp_max.loc[l, 'max'] = df_frcst[df_frcst['Datetime'].dt.year == k]['Forecast'].max()

单独执行筛选逻辑时，返回空Series：

df_frcst[df_frcst['Datetime'].dt.year == k]['Forecast']

输出：

Series([], Name: Forecast, dtype: float64)

当前程序输出的df_bp_max中，max和min列全为NaN：

==============================================================
  tahun  max  min
0  2024  NaN  NaN
1  2025  NaN  NaN
2  2026  NaN  NaN
3  2027  NaN  NaN
4  2028  NaN  NaN
5  2029  NaN  NaN
6  2030  NaN  NaN
7  2031  NaN  NaN
8  2032  NaN  NaN
9  2033  NaN  NaN

完整代码 #

import pandas as pd 
import time
import datetime
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import minimize_scalar

start_time = time.time()

pct_select = 0.98

#* data forecast
df_frcst = pd.read_csv('lstm_forecast_results.csv')
df_frcst['Datetime'] = pd.to_datetime(df_frcst['Datetime'])
df_frcst_simple = df_frcst[['Datetime', 'Forecast']]
df_frcst_simple = df_frcst_simple.rename({'Datetime':'waktu', 'Forecast':'bp' }, axis = 1)


df_bp_max = pd.DataFrame()
for k in range (2024, 2034, 1):
    l = k - 2024
    df_bp_max.loc[l, 'tahun'] = str(k)
    df_bp_max.loc[l, 'max'] = df_frcst[df_frcst['Datetime'].dt.year == k]['Forecast'].max()
    df_bp_max.loc[l, 'min'] = df_frcst[df_frcst['Datetime'].dt.year == k]['Forecast'].min()
#df_bp_max['LF'] = df_bp_max['min']/df_bp_max['max']
df_bp_max['tahun'] = df_bp_max['tahun'].apply(str)
#df_bp_max = df_bp_max.set_index('tahun')

print("==============================================================")
print(df_bp_max)

输入文件示例 #

Datetime,Actual,Forecast
2022-01-01 12:30:00,17809.0,17343.484
2022-01-01 13:00:00,17772.61,17382.861
2022-01-01 13:30:00,17867.8,17414.637
2022-01-01 14:00:00,17773.68,17504.357
2022-01-01 14:30:00,17869.88,17530.559
2022-01-01 15:00:00,17786.7,17592.822
2022-01-01 15:30:00,17943.11,17626.775
2022-01-01 16:00:00,18125.29,17686.678
2022-01-01 16:30:00,18463.05,17760.666
2022-01-01 17:00:00,18786.99,17892.475
2022-01-01 17:30:00,19238.97,18048.4
2022-01-01 18:00:00,...,...
......

解决方案 #

1. 排查数据年份范围 #

从输入文件可见，现有数据仅包含2022年记录，但代码循环的年份是2024-2033，无匹配数据自然返回空Series，max()/min()计算结果为NaN。

先验证数据中的年份分布：

# 查看数据包含的所有年份
print(df_frcst['Datetime'].dt.year.unique())

若输出无2024-2033，说明输入文件确实没有对应年份数据，需确认数据源。

2. 优化代码逻辑（替代循环，更高效） #

即使数据正确，原循环写法效率较低，推荐用pandas分组聚合：

# 按年份分组计算最大、最小值
df_bp_max = df_frcst.groupby(df_frcst['Datetime'].dt.year)['Forecast'].agg(['max', 'min']).reset_index()
# 重命名列
df_bp_max.rename(columns={'Datetime': 'tahun'}, inplace=True)
# 转换年份为字符串
df_bp_max['tahun'] = df_bp_max['tahun'].astype(str)
# 筛选目标年份范围（可选）
df_bp_max = df_bp_max[df_bp_max['tahun'].isin([str(y) for y in range(2024, 2034)])]

3. 强制保留目标年份（含缺失数据） #

若需要强制显示2024-2033所有年份，即使无数据，可先构建年份框架再合并：

# 构建目标年份的DataFrame
target_years = pd.DataFrame({'tahun': [str(y) for y in range(2024, 2034)]})
# 分组聚合已有数据的统计结果
year_stats = df_frcst.groupby(df_frcst['Datetime'].dt.year.astype(str))['Forecast'].agg(['max', 'min']).reset_index()
# 合并，缺失值自动填充NaN（可根据需求用fillna替换为0或其他值）
df_bp_max = pd.merge(target_years, year_stats, on='tahun', how='left')

内容的提问来源于stack exchange，提问作者Pandaa