如何用Python Datetime合并跨年的第52周与第0周数据?
Pandas跨年周分组拆分问题的解决方案
我有一个存储日度数据的Pandas DataFrame,尝试按周编号分组对部分列求和时发现:当年份并非从周日开始时,跨年的周数据会被拆分为两组,无法正常合并求和。
原实现代码
原周编号生成及分组求和代码如下:
df = pd.DataFrame([ {'Date': '2020-03-01', 'population': 100, 'admission': 10, 'release': 10, 'feature_id': 'abc'}, {'Date': '2020-03-02', 'population': 100, 'admission': 10, 'release': 10, 'feature_id': 'abc'}, ... ]) df['week_num'] = df['Date'].dt.strftime('%Y-%U') by_week = df.groupby(["week_num", "feature_id"])[["population", "admission", "release"]].sum().reset_index()
问题现象
分组后的数据出现跨年周被拆分的情况,示例如下:
..., { 'week_num': '2020-52', 'start_of_week': '2020-12-27T00:00:00.000Z', 'end_of_week': '2021-01-02T00:00:00.000Z', 'population': 1000, 'admission': 100, 'release': 150 }, { 'week_num': '2021-00', 'start_of_week': '2020-12-27T00:00:00.000Z', 'end_of_week': '2021-01-02T00:00:00.000Z', 'population': 200, 'admission': 120, 'release': 90 }, ...
实际上,2020-12-27至2021-01-02这一周的数据应合并为一组,即population=1000+200=1200, admission=100+120=220, release=150+90=240。
问题根源
问题出在提取周起止日期的Week类实现上,该类使用%U格式(以周日为一周起始的周编号规则),导致跨年周被拆分到两个年份的周编号中:
import datetime as dt class Week: def __init__(self, year, week_num): week_start = str(year) + " " + str(week_num) + " 0" week_end = str(year) + " " + str(week_num) + " 6" self.first_date = dt.datetime.strptime(week_start, '%Y %U %w') self.last_date = dt.datetime.strptime(week_end, '%Y %U %w') ... by_week["start_of_week"] = by_week["iso_week_num"].apply(lambda w: Week(*w.split("-")).first_date)
最终解决方案
采用ISO标准周编号格式(%G-%V,以周一为一周起始,跨年周统一归属到包含周四的年份),可以避免该问题,代码如下:
import datetime df["week_num"] = df["Date"].dt.strftime("%G-%V") by_week = df.groupby(["week_num", "feature_id"])[["population", "admission", "release"]].sum().reset_index() by_week["start_of_week"] = by_week["week_num"].apply(lambda d: datetime.strptime(d+"1", "%G-%V%u")) by_week["end_of_week"] = by_week["week_num"].apply(lambda d: datetime.strptime(d+"7", "%G-%V%u"))
内容的提问来源于stack exchange,提问作者OJT
相关产品推荐
相关产品推荐

