为何我的Binary PSO特征选择无进展？附代码与运行环境

问题描述 #

我编写了用于特征选择的Python脚本，流程如下：

• 读取路径为/content/drive/MyDrive/dataset.csv的数据集（6.62GB，含1079134行、1029列）
• 将数据划分为25个数据块
• 逐块处理：移除前3列，过滤目标列值非{A,B,C}的行，将目标值映射为{0,1,2}，提取特征矩阵X
• 结合Binary PSO（粒子群优化）与Logistic Regression分类器进行特征选择
• 所有数据块处理完成后，通过投票机制确定最优特征并保存至文件

分别在Google Colab免费版（28GB硬盘、12GB内存）及GeForce GTX 780环境下运行该脚本，均无进展，仅输出优化初始化信息，进度始终停留在0%。

完整代码：

import pandas as pd
import numpy as np
from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
import pyswarms as ps
from datetime import datetime as dt
import gc

# File path
file = '/content/drive/MyDrive/dataset.csv'
output_file = '/content/drive/MyDrive/feature_selection_output.txt'

# Constants
chunk_size = 1079134 // 25

# Initialize variables
selected_features = []

# Function to map target values
def map_target_values(val):
    mapping = {'A': 0, 'B': 1, 'C': 2}
    return mapping.get(val, -1)

# Function to apply particle swarm optimization
def f_per_particle(m, alpha, X, y, total_features, classifier):
    if np.count_nonzero(m) == 0:
        X_subset = X
    else:
        X_subset = X[:, m == 1]
    
    scores = cross_val_score(classifier, X_subset, y, cv=3)
    P = scores.mean()
    
    j = (alpha * (1.0 - P) + (1.0 - alpha) * (1 - (X_subset.shape[1] / total_features)))
    return j

def f(x, alpha, X, y, classifier):
    n_particles = x.shape[0]
    total_features = X.shape[1]
    j = [f_per_particle(x[i], alpha, X, y, total_features, classifier) for i in range(n_particles)]
    return np.array(j)

# Read and process file in chunks
for chunk in pd.read_csv(file, chunksize=chunk_size):
    # Remove the first three columns
    df = chunk.iloc[:, 3:]
    
    # Filter rows based on the left-most column values
    df = df[df.iloc[:, 0].isin(['A', 'B', 'C'])]
    
    # Map target column and extract features
    y = df.iloc[:, 0].map(map_target_values).values
    X = df.iloc[:, 1:].values
    
    # Handle missing values by imputing with zero
    imputer = SimpleImputer(strategy='constant', fill_value=0)
    X = imputer.fit_transform(X)
    
    # Scale the data
    scaler = StandardScaler()
    X = scaler.fit_transform(X)
    
    # Define classifier with increased max_iter
    classifier = LogisticRegression(max_iter=5000)
    
    # Initialize swarm for PSO
    options = {'c1': 1, 'c2': 1, 'w': 0.5, 'k': 100, 'p': 20}
    dimensions = X.shape[1]
    
    optimizer = ps.discrete.BinaryPSO(n_particles=100, dimensions=dimensions, options=options)
    
    # Perform optimization
    cost, pos = optimizer.optimize(f, iters=100, alpha=0.9, X=X, y=y, classifier=classifier)
    
    # Record selected features
    selected_features.append(pos)
    
    # Free memory
    del df, X, y, optimizer
    gc.collect()

# Use voting to select best features
final_selected_features = np.sum(selected_features, axis=0)
selected_feature_indices = np.where(final_selected_features > (len(selected_features) / 2))[0]

# Save selected features to file
with open(output_file, 'w') as f:
    for idx in selected_feature_indices:
        f.write(f"{idx}\n")

print(f"Selected features have been saved to {output_file}")

运行输出：

2024-07-06 19:24:47,269 - pyswarms.discrete.binary - INFO - Optimize for 100 iters with {'c1': 1, 'c2': 1, 'w': 0.5, 'k': 100, 'p': 20}
pyswarms.discrete.binary:   0%|          |0/100

核心问题分析 #

脚本停滞的根源是计算量远超当前硬件承载能力，具体体现在：

• PSO参数设置过于激进：100个粒子对应1026个特征（原1029列移除前3列后，特征列共1026个），每个粒子每次迭代都要执行3折交叉验证的逻辑回归训练，单次迭代计算量就已超出硬件负荷
• 逻辑回归开销过大：面对千级特征+数万行数据（每个chunk约43k行），max_iter=5000的设置会让模型训练耗时极长，叠加交叉验证和PSO的多粒子循环后，直接导致硬件算力耗尽，进程卡住

针对性优化方案 #

1. 压缩PSO计算规模 #

• 减少粒子数量：将n_particles从100降至10-20，优先保证迭代能推进
• 降低迭代次数：将iters从100降至20-30，先验证流程可行性，再逐步调整
• 简化PSO参数：k（邻域大小）设为和粒子数一致即可，无需设100；p改为2（欧氏距离），减少计算复杂度

2. 优化逻辑回归训练 #

• 启用并行计算：给LogisticRegression添加n_jobs=-1参数，利用多核CPU加速
• 降低迭代上限：先尝试max_iter=1000，若模型能收敛则进一步降低，避免不必要的迭代
• 替换轻量模型：如果精度要求允许，改用SGDClassifier（随机梯度下降）替代逻辑回归，训练速度可提升数倍

3. 预处理阶段降维 #

• 过滤低方差特征：添加VarianceThreshold移除方差极低的特征，直接减少后续所有步骤的计算量
• 优化内存占用：读取CSV时指定dtype，比如用float32替代默认的float64，降低内存消耗

4. 添加进度监控 #

• 在f_per_particle函数中添加打印语句，输出当前粒子的特征数量、交叉验证分数，确认函数是否正常执行
• 启用pyswarms的verbose模式：初始化optimizer时添加verbose=3，查看详细进度日志

优化后示例代码片段 #

# 优化后的PSO初始化
options = {'c1': 1, 'c2': 1, 'w': 0.5, 'k': 20, 'p': 2}
optimizer = ps.discrete.BinaryPSO(n_particles=20, dimensions=dimensions, options=options, verbose=3)

# 优化后的逻辑回归
classifier = LogisticRegression(max_iter=1000, n_jobs=-1)

# 添加低方差过滤
from sklearn.feature_selection import VarianceThreshold
selector = VarianceThreshold(threshold=0.01)
X = selector.fit_transform(X)

内容的提问来源于stack exchange，提问作者user366312