Python循环随机采样并合并连续数据输出的实现方法

我多次运行以下代码时，由于用到了df.sample()函数，每次输出结果都不一样。最终输出的是一个2行750列的连续数据DataFrame，这个结果来自末尾的计算循环。请问怎么把这段代码改成循环执行100次，并将每次输出的DataFrame合并保存？

df1 = dfs.sample(n=len(df_sample[df_sample.Column_name== 'Category']),replace=True)
df2 = dfs2.sample(n=len(df_sample[df_sample.Column_name== 'Category']),replace=True)
df3 = dfs3.sample(n=len(df_sample[df_sample.Column_name== 'Category']),replace=True)
all_dfs = [df1, df2, df3]
df4 = pd.concat(all_dfs, ignore_index=True)
#shp_gpd is a geopandas dataframe 
#creates a column to merge  df with geodataframe
df4['tmp'] = np.arange(len(df3))
shp_gpd['tmp'] = np.arange(len(shp_gpd))
df  = df_gpd.merge(df3, on = "tmp", how='left') 
gdf = gpd.GeoDataFrame(df)
    
#Calculations 
collist = list(gdf.columns)
emptydict = []
for j in collist[14:]:
    B1 = (((gdf.groupby(['Category1'])['Category2'].sum())/(gdf['Category2'].sum())) *  
            (gdf.groupby(['Category1'])[j].mean())).sum()
    res = {j:B1}
    emptydict.append(res)
resdf = pd.DataFrame(emptydict)
resdf1 = resdf.apply(lambda x: pd.Series(x.dropna().values))
resdf1

#standard deviation by for previous equation
collist1 = list(gdf.columns)
emptydict1 = []
for k in collist1[14:]:
    SD = (((gdf.groupby(['Category1'])['Category2'].sum())/(gdf['Category2'].sum())) * 
            (gdf.groupby(['Category1'])[k].std())).sum()
    res  = {k:SD}
    emptydict1.append(res)
SDdf = pd.DataFrame(emptydict1)
SDdf1 = SDdf.apply(lambda x: pd.Series(x.dropna().values))

df5 = [resdf1, SDdf1]
df6 = pd.concat(df5)
df6
#print(df6)

1. 封装单次运行逻辑为函数 #

将原代码中从采样到生成结果的完整流程封装成函数，方便循环调用：

import pandas as pd
import geopandas as gpd
import numpy as np

def run_single_simulation():
    # 统一获取采样规模
    sample_size = len(df_sample[df_sample.Column_name == 'Category'])
    
    # 执行采样
    df1 = dfs.sample(n=sample_size, replace=True)
    df2 = dfs2.sample(n=sample_size, replace=True)
    df3 = dfs3.sample(n=sample_size, replace=True)
    df4 = pd.concat([df1, df2, df3], ignore_index=True)
    
    # 合并地理数据
    df4['tmp'] = np.arange(len(df3))
    shp_gpd['tmp'] = np.arange(len(shp_gpd))
    merged_df = df_gpd.merge(df3, on="tmp", how='left') 
    gdf = gpd.GeoDataFrame(merged_df)
    
    # 计算均值指标
    target_cols = list(gdf.columns)[14:]
    mean_results = []
    for col in target_cols:
        cat_sum = gdf.groupby('Category1')['Category2'].sum()
        total_sum = gdf['Category2'].sum()
        cat_mean = gdf.groupby('Category1')[col].mean()
        calc_val = (cat_sum / total_sum * cat_mean).sum()
        mean_results.append({col: calc_val})
    mean_df = pd.DataFrame(mean_results).apply(lambda x: pd.Series(x.dropna().values))
    
    # 计算标准差指标
    std_results = []
    for col in target_cols:
        cat_sum = gdf.groupby('Category1')['Category2'].sum()
        total_sum = gdf['Category2'].sum()
        cat_std = gdf.groupby('Category1')[col].std()
        calc_val = (cat_sum / total_sum * cat_std).sum()
        std_results.append({col: calc_val})
    std_df = pd.DataFrame(std_results).apply(lambda x: pd.Series(x.dropna().values))
    
    # 合并均值和标准差结果
    combined_df = pd.concat([mean_df, std_df])
    return combined_df

2. 循环执行100次并收集结果 #

创建空列表存储每次模拟的输出，循环调用函数后合并所有结果：

# 初始化结果容器
all_simulations = []

# 执行100次模拟
for sim_id in range(100):
    result = run_single_simulation()
    # 添加模拟ID列，区分不同次的结果
    result['simulation_id'] = sim_id + 1
    all_simulations.append(result)

# 合并所有结果为一个DataFrame
final_result = pd.concat(all_simulations, ignore_index=True)

3. 保存合并后的结果 #

将最终结果保存为CSV或Excel文件：

# 保存为CSV文件
final_result.to_csv('100_simulations_output.csv', index=False)

# 保存为Excel文件（需安装openpyxl库）
final_result.to_excel('100_simulations_output.xlsx', index=False)

优化说明 #

• 复用变量：统一使用target_cols代替原代码中的collist和collist1，减少重复定义。
• 提取重复计算：将分组求和、总求和等重复计算的部分单独提取，提升代码效率。
• 添加模拟标识：通过simulation_id列标记每次模拟的序号，后续分析时可按此分组统计均值、标准差等指标。

内容的提问来源于stack exchange，提问作者domarom