循环导出pandas DataFrame列表为CSV时仅最后一个文件保存问题
问题原因
所有导出的CSV使用了完全相同的文件名,导致前序导出的文件被后续内容反复覆盖,最终仅保留最后一个DataFrame的导出结果,具体代码问题如下:
county_id变量在遍历dfs生成new_dfs的循环中被重复赋值,每次循环都会覆盖上一次的取值,循环结束后county_id仅会保留最后一个DataFrame对应的取值。- 后续遍历
new_dfs导出CSV时,所有文件名都使用了最终的同一个county_id和固定的date参数,生成的文件路径完全一致,自然会出现覆盖问题。
修复方案
步骤1:存储每个DataFrame对应的county_id
新增列表存储每个DataFrame对应的county_id,保证和new_dfs中的元素一一对应,同时可以优化原地址字符替换的逻辑,用pandas向量化操作替代循环,运行效率更高:
new_dfs = [] county_ids = [] # 新增列表存储对应county_id for df in dfs: # 原有构造new_df的逻辑保持不变 new_df = pd.DataFrame() new_df['Original Addr string'] = df['StreetConc'] new_df['Addr #'] = df['AddNum'] new_df['Prefix'] = df['StPreDir'] new_df['Street Name'] = df['StName'] new_df['StreetType'] = df['StType'] new_df['Suffix'] = df['StDir'] new_df['Multi-Unit'] = '' new_df['City'] = df['City'] new_df['Zip Code'] = df['PostCode'] new_df['4'] = df['PostalExt'] new_df['County'] = df['CountyID'] new_df['Addr Type'] = '' new_df['Precint Part Name'] = '' new_df['Lat'] = df['X'] new_df['Long'] = df['Y'] # 优化替换逗号的逻辑,无需循环遍历行 new_df['Original Addr string'] = new_df['Original Addr string'].str.replace(',', ' ') county_id = df.iloc[0, 37] new_dfs.append(new_df) county_ids.append(county_id) # 存入对应county_id
步骤2:导出时使用对应county_id生成唯一文件名
遍历导出时,取对应位置的county_id生成文件名,还可以额外加上序号i,保证即使多个DataFrame的county_id重复也不会出现覆盖问题:
for i in range(len(new_dfs)): save_path = f'{Output}/ADDR_{county_ids[i]}_{i}_{date}.csv' new_dfs[i].to_csv(save_path, index=False)
内容的提问来源于stack exchange,提问作者MulfordnSons
相关产品推荐
相关产品推荐

