使用粒子群优化(PSO)优化随机森林回归模型遇UFuncTypeError求助

问题根源 #

你遇到的UFuncTypeError本质是代码里几个核心逻辑错误导致的：

• 适应度函数返回字符串而非数值：fitness_function返回格式化字符串f'The error is: {error}'，但PSO需要数值型适应度值来执行np.argmin、np.average这类计算，字符串无法参与数值运算，直接触发类型不匹配错误。
• 粒子维度完全颠倒：初始化particles时写成了[维度数, 种群数]的形状，但实际应该是[种群数, 维度数]，导致后续位置、速度更新时数组形状不兼容。
• 未利用PSO参数优化模型：代码始终用初始训练好的RFR模型，PSO生成的粒子参数完全没用来调整模型，失去了优化意义。
• 参数范围设置错误：你给出的position_min/max对应24个自变量维度，但PSO应该优化的是随机森林的超参数（比如树的数量、最大深度等），不是特征值。

修复方案 #

1. 重新定义适应度函数 #

让函数接收PSO生成的超参数，用这些参数训练随机森林，返回数值型MSE作为适应度值：

def fitness_function(params):
    # 解析PSO传入的超参数（整数型参数需取整）
    n_estimators = int(params[0])
    max_depth = int(params[1])
    max_features = params[2]
    
    # 用当前参数训练模型
    model = RandomForestRegressor(n_estimators=n_estimators, 
                                  max_depth=max_depth, 
                                  max_features=max_features,
                                  random_state=1)
    model.fit(X_train, y_train)
    
    # 计算测试集MSE作为适应度（值越小越好）
    y_pred = model.predict(X_test)
    return mean_squared_error(y_test, y_pred)

2. 修正粒子初始化逻辑 #

确保粒子形状为[种群数量, 超参数维度]，同时合理设置超参数的取值范围：

# 示例：优化3个核心超参数
param_bounds = {
    'n_estimators': (50, 200),   # 决策树数量
    'max_depth': (3, 20),        # 树最大深度
    'max_features': (0.1, 1.0)   # 每棵树使用的特征比例
}
position_min = [v[0] for v in param_bounds.values()]
position_max = [v[1] for v in param_bounds.values()]
dimension = len(param_bounds)

3. 修复PSO主函数的维度错误 #

调整粒子、速度的初始化形状，修正迭代逻辑：

def pso_2d(population, dimension, position_min, position_max, generation, fitness_criterion):
    # 初始化粒子：[种群数量, 参数维度]
    particles = np.array([np.random.uniform(position_min[i], position_max[i], population) 
                          for i in range(dimension)]).T
    
    # 初始化个体最优位置和适应度
    pbest_position = particles.copy()
    pbest_fitness = np.array([fitness_function(p) for p in particles])
    
    # 初始化全局最优位置和适应度
    gbest_index = np.argmin(pbest_fitness)
    gbest_position = pbest_position[gbest_index].copy()
    gbest_fitness = pbest_fitness[gbest_index]
    
    # 初始化速度：与粒子形状一致
    velocity = np.zeros_like(particles)
    
    # PSO迭代过程
    for t in range(generation):
        # 遍历每个粒子更新速度和位置
        for n in range(population):
            velocity[n] = update_velocity(particles[n], velocity[n], pbest_position[n], gbest_position)
            particles[n] = update_position(particles[n], velocity[n])
            
            # 确保参数在设定范围内
            particles[n] = np.clip(particles[n], position_min, position_max)
        
        # 更新个体最优
        current_fitness = np.array([fitness_function(p) for p in particles])
        update_mask = current_fitness < pbest_fitness
        pbest_position[update_mask] = particles[update_mask].copy()
        pbest_fitness[update_mask] = current_fitness[update_mask].copy()
        
        # 更新全局最优
        current_gbest_index = np.argmin(pbest_fitness)
        current_gbest_fitness = pbest_fitness[current_gbest_index]
        if current_gbest_fitness < gbest_fitness:
            gbest_position = pbest_position[current_gbest_index].copy()
            gbest_fitness = current_gbest_fitness
        
        # 提前终止条件
        if gbest_fitness <= fitness_criterion:
            break
    
    # 输出结果
    print('全局最优超参数: ', gbest_position)
    print('最优适应度(MSE): ', gbest_fitness)
    print('迭代代数: ', t+1)
    return gbest_position, gbest_fitness

4. 修正速度和位置更新函数 #

改用向量运算替代循环，确保操作都是数值型：

def update_velocity(particle, velocity, pbest, gbest, w_min=0.5, w_max=1.0, c=0.1):
    r1 = random.uniform(0, 1)
    r2 = random.uniform(0, 1)
    w = random.uniform(w_min, w_max)
    c1 = c
    c2 = c
    
    # 向量运算，无需逐元素循环
    new_velocity = w * velocity + c1*r1*(pbest - particle) + c2*r2*(gbest - particle)
    return new_velocity

def update_position(particle, velocity):
    new_particle = particle + velocity
    return new_particle

完整可运行代码 #

import numpy as np
import pandas as pd
import random
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.ensemble import RandomForestRegressor

# 示例数据集（替换为你的真实数据）
np.random.seed(1)
X = np.random.rand(1000, 24)
y = np.random.rand(1000)

# 划分训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=1)

# 适应度函数
def fitness_function(params):
    n_estimators = int(params[0])
    max_depth = int(params[1])
    max_features = params[2]
    
    model = RandomForestRegressor(n_estimators=n_estimators, 
                                  max_depth=max_depth, 
                                  max_features=max_features,
                                  random_state=1)
    model.fit(X_train, y_train)
    
    y_pred = model.predict(X_test)
    return mean_squared_error(y_test, y_pred)

# 速度更新函数
def update_velocity(particle, velocity, pbest, gbest, w_min=0.5, w_max=1.0, c=0.1):
    r1 = random.uniform(0, 1)
    r2 = random.uniform(0, 1)
    w = random.uniform(w_min, w_max)
    c1 = c
    c2 = c
    
    new_velocity = w * velocity + c1*r1*(pbest - particle) + c2*r2*(gbest - particle)
    return new_velocity

# 位置更新函数
def update_position(particle, velocity):
    new_particle = particle + velocity
    return new_particle

# PSO主函数
def pso_2d(population, dimension, position_min, position_max, generation, fitness_criterion):
    particles = np.array([np.random.uniform(position_min[i], position_max[i], population) 
                          for i in range(dimension)]).T
    
    pbest_position = particles.copy()
    pbest_fitness = np.array([fitness_function(p) for p in particles])
    
    gbest_index = np.argmin(pbest_fitness)
    gbest_position = pbest_position[gbest_index].copy()
    gbest_fitness = pbest_fitness[gbest_index]
    
    velocity = np.zeros_like(particles)
    
    for t in range(generation):
        for n in range(population):
            velocity[n] = update_velocity(particles[n], velocity[n], pbest_position[n], gbest_position)
            particles[n] = update_position(particles[n], velocity[n])
            particles[n] = np.clip(particles[n], position_min, position_max)
        
        current_fitness = np.array([fitness_function(p) for p in particles])
        update_mask = current_fitness < pbest_fitness
        pbest_position[update_mask] = particles[update_mask].copy()
        pbest_fitness[update_mask] = current_fitness[update_mask].copy()
        
        current_gbest_index = np.argmin(pbest_fitness)
        current_gbest_fitness = pbest_fitness[current_gbest_index]
        if current_gbest_fitness < gbest_fitness:
            gbest_position = pbest_position[current_gbest_index].copy()
            gbest_fitness = current_gbest_fitness
        
        if gbest_fitness <= fitness_criterion:
            break
    
    print('全局最优超参数: ', gbest_position)
    print('最优适应度(MSE): ', gbest_fitness)
    print('迭代代数: ', t+1)
    return gbest_position, gbest_fitness

# 超参数范围定义
param_bounds = {
    'n_estimators': (50, 200),
    'max_depth': (3, 20),
    'max_features': (0.1, 1.0)
}
position_min = [v[0] for v in param_bounds.values()]
position_max = [v[1] for v in param_bounds.values()]

# 运行PSO优化
pso_2d(population=50, 
       dimension=len(param_bounds), 
       position_min=position_min, 
       position_max=position_max, 
       generation=100, 
       fitness_criterion=1e-3)

关键说明 #

• 现在PSO真正在优化随机森林的超参数，而非无意义的特征值范围
• 适应度函数返回数值型MSE，彻底解决了类型错误问题
• 粒子维度和形状完全匹配，避免了数组操作的维度冲突

内容的提问来源于stack exchange，提问作者Swati Singh