如何简化基于多变量的groupby()重复数据分析流程

问题背景 #

我从一份大型调查数据集生成了多个DataFrame，希望通过三个不同变量使用groupby()进行数据分析，每次得到不同的数据集。当前通过复制粘贴代码并替换变量/DataFrame名称来完成操作（示例代码如下），想找到避免重复构建DataFrame的方法：

# 针对Location变量的实现
Fulltime_art_Location = AWAW[['art_full_time', 'Location']].value_counts()
Fulltime_art_Location_table = Fulltime_art_Location.to_frame()
Fulltime_art_Location_pivot = Fulltime_art_Location_table.pivot_table(index=['Location'], columns=['art_full_time'])

Fulltime_art_Location_pivot_percent = Fulltime_art_Location_pivot.apply(lambda x: x / x.sum(), axis=1)
Fulltime_art_Location_pivot_percent

# 针对Age变量的实现
Fulltime_art_age = AWAW[['art_full_time', 'Age']].value_counts()
Fulltime_art_age_table = Fulltime_art_age.to_frame()
Fulltime_art_age_pivot = Fulltime_art_age_table.pivot_table(index=['Age'], columns=['art_full_time'])

Fulltime_art_age_pivot_percent = Fulltime_art_age_pivot.apply(lambda x: x / x.sum(), axis=1)
Fulltime_art_age_pivot_percent

# 针对Race变量的实现
Fulltime_art_race = AWAW[['art_full_time', 'Race']].value_counts()
Fulltime_art_race_table = Fulltime_art_race.to_frame()
Fulltime_art_race_pivot = Fulltime_art_race_table.pivot_table(index=['Race'], columns=['art_full_time'])

Fulltime_art_race_pivot_percent = Fulltime_art_race_pivot.apply(lambda x: x / x.sum(), axis=1)
Fulltime_art_race_pivot_percent

优化方案 #

1. 封装可复用函数 #

把重复的逻辑封装成函数，只需传入目标变量名即可完成计算，彻底避免复制粘贴：

def calculate_fulltime_pivot(df, target_col):
    # 统计变量与art_full_time的组合频次
    count_data = df[['art_full_time', target_col]].value_counts()
    # 转换为DataFrame格式
    count_df = count_data.to_frame()
    # 生成透视表
    pivot_df = count_df.pivot_table(index=[target_col], columns=['art_full_time'])
    # 计算每行的占比
    pivot_percent = pivot_df.apply(lambda x: x / x.sum(), axis=1)
    return pivot_percent

# 调用函数处理三个变量
location_pivot = calculate_fulltime_pivot(AWAW, 'Location')
age_pivot = calculate_fulltime_pivot(AWAW, 'Age')
race_pivot = calculate_fulltime_pivot(AWAW, 'Race')

2. 循环批量处理并存储结果 #

如果需要一次性处理多个变量，可以用循环遍历目标变量列表，将结果存入字典统一管理：

# 定义需要分析的变量列表
target_columns = ['Location', 'Age', 'Race']
# 用字典存储每个变量的分析结果
pivot_results = {}

for col in target_columns:
    count_data = AWAW[['art_full_time', col]].value_counts()
    count_df = count_data.to_frame()
    pivot_df = count_df.pivot_table(index=[col], columns=['art_full_time'])
    pivot_percent = pivot_df.apply(lambda x: x / x.sum(), axis=1)
    pivot_results[col] = pivot_percent

# 调用结果示例
# print(pivot_results['Location'])
# print(pivot_results['Age'])
# print(pivot_results['Race'])

3. 简化代码逻辑（进阶版） #

可以用unstack()替代pivot_table，同时用div()替换apply(lambda...)，让代码更简洁高效：

def calculate_fulltime_pivot(df, target_col):
    # 统计频次并直接转换为透视表格式（空值填充为0）
    pivot_df = df[['art_full_time', target_col]].value_counts().unstack(fill_value=0)
    # 计算行占比，div方法比apply更高效
    pivot_percent = pivot_df.div(pivot_df.sum(axis=1), axis=0)
    return pivot_percent

内容的提问来源于stack exchange，提问作者AnnB