使用Hyperopt进行XGBoost贝叶斯调参时遇float转整数错误求助

问题场景 #

使用Hyperopt对XGBoost回归器进行贝叶斯超参数调优时，执行best = fmin(fn=objective, ...)代码行触发错误：'float' object cannot be interpreted as an integer。

错误原因 #

1. hp.quniform返回浮点数：n_estimators、min_child_weight通过hp.quniform定义，该函数返回浮点数，但XGBoost要求这两个参数必须是整数类型。尽管后续提取最优参数时做了类型转换，但在objective函数初始化XGBRegressor时，传入的参数仍是浮点数，直接触发类型错误。
2. hp.randint返回numpy整数：部分版本中hp.randint返回numpy整数类型，XGBoost可能无法直接识别，需转为Python原生整数。

修复方案 #

方案一：在objective函数内强制转换参数类型 #

在objective函数中，对需要整数类型的参数先做类型转换，再传入XGBRegressor：

def objective(params):
    # 强制转换整数类型参数
    params['n_estimators'] = int(params['n_estimators'])
    params['max_depth'] = int(params['max_depth'])
    params['min_child_weight'] = int(params['min_child_weight'])
    
    model = xgb.XGBRegressor(**params)
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)
    score = -r2_score(y_test, y_pred)
    return {'loss': score, 'status': STATUS_OK}

方案二：改用hp.qint生成整数搜索空间（推荐） #

如果Hyperopt版本≥0.2.7，直接使用hp.qint替代hp.quniform，该函数会直接生成整数类型的参数值，无需额外转换：

param_space = {
    'n_estimators': hp.qint('n_estimators', 100, 300, 1),
    'max_depth': hp.randint('max_depth', 3, 6),
    'learning_rate': hp.uniform('learning_rate', 0.01, 0.2),
    'min_child_weight': hp.qint('min_child_weight', 1, 3, 1),
    'reg_alpha': hp.uniform('reg_alpha', 0, 1),
    'reg_lambda': hp.uniform('reg_lambda', 0, 1)
}

同时仍需在objective函数中对max_depth做类型转换，避免numpy整数兼容问题：

def objective(params):
    params['max_depth'] = int(params['max_depth'])
    model = xgb.XGBRegressor(**params)
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)
    score = -r2_score(y_test, y_pred)
    return {'loss': score, 'status': STATUS_OK}

完整修正代码（方案一） #

# Define the objective function
def objective(params):
    # 转换整数类型参数
    params['n_estimators'] = int(params['n_estimators'])
    params['max_depth'] = int(params['max_depth'])
    params['min_child_weight'] = int(params['min_child_weight'])
    
    model = xgb.XGBRegressor(**params)
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)
    score = -r2_score(y_test, y_pred)  # Negative R2 score for minimization
    return {'loss': score, 'status': STATUS_OK}

# Define the search space for hyperparameters
param_space = {
    'n_estimators': hp.quniform('n_estimators', 100, 300, 1),
    'max_depth': hp.randint('max_depth', 3, 6),  # Use hp.randint for integer choices
    'learning_rate': hp.uniform('learning_rate', 0.01, 0.2),
    'min_child_weight': hp.quniform('min_child_weight', 1, 3, 1),
    'reg_alpha': hp.uniform('reg_alpha', 0, 1),
    'reg_lambda': hp.uniform('reg_lambda', 0, 1)
}

# Initialize Hyperopt Trials
trials = Trials()

# Set a random seed for Hyperopt
np.random.seed(42)

# Perform Bayesian hyperparameter tuning
best = fmin(fn=objective,
            space=param_space,
            algo=tpe.suggest,
            max_evals=50,  # Number of optimization iterations
            trials=trials)

# Get the best hyperparameters from the optimization
best_n_estimators = int(best['n_estimators'])
best_max_depth = int(best['max_depth'])
best_learning_rate = best['learning_rate']
best_min_child_weight = int(best['min_child_weight'])
best_reg_alpha = best['reg_alpha']
best_reg_lambda = best['reg_lambda']

# Train the final model with the best hyperparameters
best_params = {
    'n_estimators': best_n_estimators,
    'max_depth': best_max_depth,
    'learning_rate': best_learning_rate,
    'min_child_weight': best_min_child_weight,
    'reg_alpha': best_reg_alpha,
    'reg_lambda': best_reg_lambda
}

final_model = xgb.XGBRegressor(**best_params)
final_model.fit(X_train, y_train)

# Make predictions on training and testing data
y_train_pred = final_model.predict(X_train)
y_test_pred = final_model.predict(X_test)

# Calculate R2 scores for training and testing data
train_r2 = r2_score(y_train, y_train_pred)
test_r2 = r2_score(y_test, y_test_pred)

# Print the best hyperparameters and R2 scores
print(f'Best Hyperparameters: {best_params}')
print(f'Training R2 Score: {train_r2:.2f}')
print(f'Testing R2 Score: {test_r2:.2f}')

内容的提问来源于stack exchange，提问作者Armaan Gohil