使用RecursiveFeatureAddition时遇List index out of range问题求助

问题重现 #

运行RecursiveFeatureAddition的fit方法时触发索引越界错误，但相同的交叉验证折（folds）在GridSearchCV中可正常使用，且已手动验证索引未超出X_train范围。

相关代码 #

初始化RecursiveFeatureAddition

rfa2_search = RecursiveFeatureAddition(mod_XG, scoring=accuracy, cv=folds, threshold=0.01)

自定义walk-forward交叉验证拆分函数

def walk_forward(df_train, n_fold, zip_):
    # non-overlapping walk forward
    
    total_points = df_train.shape[0] # 5000
    
    sub_val_points = int(0.5 * total_points / n_fold) # 500
    sub_train_points = int(total_points - sub_val_points * n_fold) # 2500

    arr_tra, arr_val = [], []
    step = sub_val_points # non overlapping criteria
    pos = 0
    
    for _ in range(n_fold):
        arr_tra.append(np.arange(pos, pos + sub_train_points))
        arr_val.append(np.arange(pos + sub_train_points, pos + sub_train_points + sub_val_points))
        
        pos += step
    
    if zip_:
        custom_cv = zip(list(arr_tra), list(arr_val))
    else:
        custom_cv = [list(arr_tra), list(arr_val)]
          
    return custom_cv

触发错误的代码

rfa2_search.fit(X_train, y_train)

错误信息：

IndexError: list index out of range

问题原因 #

1. Zip对象的一次性迭代特性：当zip_=True时返回的zip对象是一次性迭代器，如果RecursiveFeatureAddition内部多次遍历交叉验证折（比如特征筛选的每一轮都需要交叉验证），第一次遍历后zip对象就会被耗尽，后续访问就会出现索引越界。而GridSearchCV仅需遍历一次，因此能正常工作。
2. 错误的cv格式（else分支）：当zip_=False时返回的[arr_tra, arr_val]不符合sklearn交叉验证接口规范——sklearn要求cv是包含(train_idx, val_idx)元组的可迭代对象，而非两个分别存储所有训练/验证索引的大列表，这种格式RecursiveFeatureAddition无法正确解析，进而导致索引错误。

解决方案 #

修改walk_forward函数，统一返回符合sklearn规范的交叉验证格式：将zip对象转换为列表，同时废弃错误的else分支格式，确保返回的是每个折的训练/验证索引配对的元组列表。

修改后的函数：

def walk_forward(df_train, n_fold):
    # non-overlapping walk forward
    
    total_points = df_train.shape[0]
    sub_val_points = int(0.5 * total_points / n_fold)
    sub_train_points = int(total_points - sub_val_points * n_fold)

    arr_tra, arr_val = [], []
    step = sub_val_points
    pos = 0
    
    for _ in range(n_fold):
        arr_tra.append(np.arange(pos, pos + sub_train_points))
        arr_val.append(np.arange(pos + sub_train_points, pos + sub_train_points + sub_val_points))
        pos += step
    
    # 统一返回(train_idx, val_idx)元组的列表，符合sklearn cv规范
    custom_cv = list(zip(arr_tra, arr_val))
    return custom_cv

验证修改 #

1. 调用函数时不再需要zip_参数，直接生成标准格式的交叉验证折：

   folds = walk_forward(df_train, n_fold=5)
   

2. 此时folds是一个列表，每个元素是(train_indices_array, val_indices_array)的元组，和sklearn自带的KFold返回格式完全一致，RecursiveFeatureAddition可以正确遍历和使用这些索引，不会再触发索引越界错误。

内容的提问来源于stack exchange，提问作者ilPaghyNonPaga