Pandas循环处理DataFrame报IndexError，求更新数据帧的解决方案

创建了两个Pandas数据帧：

mu, sigma = 4., 1.3 
s1 = pd.DataFrame(np.random.lognormal(mu, sigma, size=(1000, 1000)))
min_value = s1.values.min() 
max_value = s1.values.max() 
s2 = pd.DataFrame(np.random.uniform(min_value, max_value, size=(1000, 1)))

其中s2仅有1列，s1包含1000列、每列1000个元素。需求为：按s1的列循环，将s2的每个元素按索引顺序与s1对应元素运算，根据条件更新s2，并将结果记录到r和c两个输出数据帧中；每次循环后，更新后的s2进入下一轮循环，遍历s1所有列。

原循环代码：

r = pd.DataFrame(columns=['R'])
c = pd.DataFrame(columns=['C'])


for col in s1.columns:
    for idx, row in enumerate(s2.values):
        if (row >= 0.95*s1.iloc[idx, col]) and (row <= 1.05*s1.iloc[idx, col]):
            s2.iloc[idx,col] = 0
            r.iloc[idx,:] = 0
        elif row > s1.iloc[idx, col]:
            diff = row - s1.iloc[idx, col]
            s2.iloc[idx,col] = diff
            r.iloc[idx,0] = diff
            c.iloc[idx,0] = s2.iloc[idx, col]
        else:
            r.iloc[idx,0] = row

result = pd.concat([s2, r, c], axis=1)

运行时出现错误：IndexError: iloc cannot enlarge its target object，希望实现单列更新r/c避免覆盖或者每次循环将结果记录到r/c的新列。

1. iloc不允许通过赋值不存在的行/列来扩展DataFrame：初始的r和c是仅含列名的空DataFrame，用r.iloc[idx,:]赋值时，idx对应的行不存在，触发报错。
2. 原代码存在逻辑错误：s2只有1列，但代码中用s2.iloc[idx,col]赋值，col是s1的列索引（0-999），会导致列索引越界。

方案1：单列更新r和c（避免覆盖） #

先初始化r和c的行数与s2一致，确保行索引存在；同时修正s2的列索引错误：

import pandas as pd
import numpy as np

mu, sigma = 4., 1.3 
s1 = pd.DataFrame(np.random.lognormal(mu, sigma, size=(1000, 1000)))
min_value = s1.values.min() 
max_value = s1.values.max() 
s2 = pd.DataFrame(np.random.uniform(min_value, max_value, size=(1000, 1)))

# 初始化r和c为1000行的空DataFrame，列名分别为'R'和'C'
r = pd.DataFrame(np.nan, index=s2.index, columns=['R'])
c = pd.DataFrame(np.nan, index=s2.index, columns=['C'])

for col in s1.columns:
    for idx, row_val in enumerate(s2.iloc[:, 0].values):
        s1_val = s1.iloc[idx, col]
        if 0.95 * s1_val <= row_val <= 1.05 * s1_val:
            # s2只有1列，固定用列索引0
            s2.iloc[idx, 0] = 0
            r.iloc[idx, 0] = 0
        elif row_val > s1_val:
            diff = row_val - s1_val
            s2.iloc[idx, 0] = diff
            r.iloc[idx, 0] = diff
            c.iloc[idx, 0] = diff
        else:
            r.iloc[idx, 0] = row_val

result = pd.concat([s2, r, c], axis=1)

方案2：每次循环新增列到r和c（记录每轮结果） #

如果需要保留每一轮循环的结果，可在每次循环时给r和c新增列，列名用当前s1的列名：

import pandas as pd
import numpy as np

mu, sigma = 4., 1.3 
s1 = pd.DataFrame(np.random.lognormal(mu, sigma, size=(1000, 1000)))
min_value = s1.values.min() 
max_value = s1.values.max() 
s2 = pd.DataFrame(np.random.uniform(min_value, max_value, size=(1000, 1)))

# 初始化空的r和c，后续逐列添加轮次结果
r = pd.DataFrame()
c = pd.DataFrame()

for col in s1.columns:
    # 临时存储当前轮的结果，列名标记轮次
    r_col = pd.Series(index=s2.index, name=f'R_{col}')
    c_col = pd.Series(index=s2.index, name=f'C_{col}')
    
    for idx, row_val in enumerate(s2.iloc[:, 0].values):
        s1_val = s1.iloc[idx, col]
        if 0.95 * s1_val <= row_val <= 1.05 * s1_val:
            s2.iloc[idx, 0] = 0
            r_col.iloc[idx] = 0
        elif row_val > s1_val:
            diff = row_val - s1_val
            s2.iloc[idx, 0] = diff
            r_col.iloc[idx] = diff
            c_col.iloc[idx] = diff
        else:
            r_col.iloc[idx] = row_val
    
    # 将当前轮结果列添加到r和c中
    r = pd.concat([r, r_col], axis=1)
    c = pd.concat([c, c_col], axis=1)

# 最终结果：s2是最后一轮更新后的值，r和c包含所有轮次的记录
result = pd.concat([s2, r, c], axis=1)

内容的提问来源于stack exchange，提问作者Giantanque