嵌套循环向数据集添加行时遇报错及迭代不全问题求助

问题根源 #

1. 遍历终止过早：原函数找到第一个匹配项后立即return，仅处理hours_date_match_barc第一行就停止，无法遍历全部行。
2. 不匹配行无返回值：data3包含compl_ref_index为FOOL的行，这些行与laborcompl_ref的组不匹配，循环结束后函数无返回值，触发ValueError。
3. 参数取值错误：n和mf错误取整个DataFrame的第一行或均值，未使用当前遍历行的对应数值，逻辑不符合需求。

修复后的代码 #

import pandas as pd
import numpy as np

pd.set_option('display.max_rows', None)

data2={
    "ID": [ "BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR"],
    "PHASENAME": [ "C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C"],
    "DAY_COUNTER": [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100],
    "DAILY_LABOR_PERCENT": [0.002,0.002,0.002,0.005,0.006,0.009,0.01,0.01,0.01,0.011,0.012,0.012,0.012,0.012,0.012,0.012,0.012,0.012,0.012,0.012,0.013,0.013,0.013,0.013,0.013,0.014,0.014,0.014,0.013,0.013,0.015,0.015,0.014,0.014,0.014,0.012,0.011,0.011,0.011,0.012,0.012,0.012,0.013,0.013,0.013,0.012,0.012,0.012,0.011,0.011,0.011,0.011,0.01,0.01,0.01,0.01,0.009,0.009,0.009,0.009,0.008,0.008,0.008,0.008,0.009,0.009,0.009,0.009,0.009,0.009,0.008,0.008,0.009,0.009,0.009,0.009,0.009,0.009,0.009,0.009,0.008,0.007,0.007,0.007,0.005,0.004,0.004,0.005,0.005,0.005,0.005,0.004,0.005,0.005,0.005,0.006,0.008,0.008,0.009,0.009],
    "compl_ref_index" : ["BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC"],
    }

data3={
    "ID": ["FOO","FOO","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR","BAR"],
    "PHASENAME": ["L","L","C","C","C","C","C","C","C","C","C","C","C","C","C","C","C"],
    "C_DAYS": [17,  17, 11, 11, 11, 11, 13, 13, 13, 13, 12, 12, 12, 12, 14, 14, 14],
    "Multi_Factor": [5.882353,5.882353,9.090909,9.090909,9.090909,9.090909,7.692308,7.692308,7.692308,7.692308,8.333333,8.333333,8.333333,8.333333,7.142857,7.142857,7.142857],
    "compl_ref_index" : ["FOOL","FOOL","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC","BARC", "BARC"]
    }

laborcompl_ref=pd.DataFrame(data2)
hours_date_match_barc=pd.DataFrame(data3)


def compress_and_process_group(group):
    results = []
    target_ref = group["compl_ref_index"].iloc[0]
    # 遍历hours_date_match_barc的每一行
    for _, row in hours_date_match_barc.iterrows():
        if row["compl_ref_index"] == target_ref:
            # 使用当前行的C_DAYS和Multi_Factor
            n = row["C_DAYS"]
            mf = int(row["Multi_Factor"])
            first_day = group["DAY_COUNTER"].iloc[0]
            
            # 生成重复的组数据并截取前n行
            rep = np.tile(group.values, (n // len(group) + 1, 1))
            out = pd.DataFrame(rep, columns=group.columns).iloc[:n]
            out["DAY_COUNTER"] = range(first_day, first_day + n)
            out["C_DAYS"] = n
            
            # 计算合并后的labor值，确保长度足够
            labor_values = group["DAILY_LABOR_PERCENT"].values
            sum_of_rows = np.add.reduceat(labor_values, range(0, len(labor_values), mf))
            # 如果sum_of_rows长度不足n，补0填充
            if len(sum_of_rows) < n:
                sum_of_rows = np.pad(sum_of_rows, (0, n - len(sum_of_rows)), mode='constant')
            selected_data = sum_of_rows[:n]
            out["NEW_LBR_COMPL"] = selected_data
            
            results.append(out)
            print(f"处理匹配项: {row['compl_ref_index']}")
    
    # 返回所有匹配结果的合并，无匹配则返回空DataFrame
    return pd.concat(results, ignore_index=True) if results else pd.DataFrame(columns=group.columns)


# 处理每个组并合并结果
result = pd.concat([compress_and_process_group(group) for _, group in laborcompl_ref.groupby(["ID", "PHASENAME"])])
print(f"最终结果行数: {len(result)}")

关键修改说明 #

• 收集所有结果：用列表results存储每一行匹配后的输出，最后合并返回，避免提前终止遍历。
• 匹配逻辑优化：直接对比当前组的compl_ref_index（分组后组内值一致，取第一个即可）和遍历行的对应值，更高效。
• 参数修正：n和mf改为使用当前遍历行的C_DAYS和Multi_Factor（转成int），符合业务逻辑。
• 空值处理：无匹配项时返回空的DataFrame，避免concat时出现错误；同时处理sum_of_rows长度不足的情况，用0填充防止索引越界。
• 移除无效代码：删掉原函数中无用的index +=1（iterrows的index自动迭代，手动修改无意义）和无效循环for i in range(0, len(out))（重复执行无实际作用）。

内容的提问来源于stack exchange，提问作者Ty Kendall