使用sklearn RandomizedSearchCV遇ValueError: Domain error的解决方法

我在对SGDClassifier的超参数执行随机搜索时，randomizedsearch_estimator.fit(x_train, y_train)无法正常运行，出现ValueError: Domain error in arguments错误。相关代码及报错堆栈信息如下：

from constants import (SPLITS_NUM, SEED, N_JOBS, PROBLEM_METRIC)  
from sklearn.linear_model import SGDClassifier
from sklearn.model_selection import (KFold, RandomizedSearchCV)
from scipy.stats import (randint, uniform)  


def randomized_search(estimator, param_distributions, x_train, y_train,
                  x_validation, y_validation):
    kfold = KFold(n_splits=SPLITS_NUM, shuffle=True, random_state=SEED)
    randomizedsearch_estimator = RandomizedSearchCV(estimator,
                                                param_distributions,
                                                cv=kfold,
                                                return_train_score=True,
                                                n_jobs=N_JOBS,
                                                scoring=PROBLEM_METRIC)
    search = randomizedsearch_estimator.fit(x_train, y_train)
    print(f"Best estimator:\n{search.best_estimator_} \
       \nBest parameters:\n{search.best_params_} \
       \nBest cross-validation score: {search.best_score_:.3f} \
       \nBest test score: {search.score(x_validation, y_validation):.3f}\n\n")

def searching_list():
    return [(SGDClassifier(random_state=SEED, learning_rate='optimal', class_weight='balanced'), {
    'alpha': uniform(0.15, 0.25),
    'l1_ratio': uniform(0.002, 0.008),
    'max_iter': randint(45000, 55000),
    'tol': uniform(0.04, 0.12),
    'epsilon': uniform(45000, 55000),
    'power_t': uniform(-100000, -50000),
    'loss': [
        'hinge', 'log_loss', 'modified_huber', 'squared_hinge', 'perceptron', 'squared_error',
        'huber', 'epsilon_insensitive', 'squared_epsilon_insensitive'
    ],
    'penalty': ['l2', 'l1', 'elasticnet']
})]

def parameter_initializer(features_train, target_train, features_validation,
                      target_validation):
    for model, distribution in searching_list():
        randomized_search(model, distribution, features_train, target_train, features_validation,
                      target_validation)

报错堆栈：

Traceback (most recent call last):
  File "c:\Users\username\Desktop\some-calculator\graph-analyzer\graph_analyzer.py", line 197, in <module>
    main()
  File "c:\Users\username\Desktop\some-calculator\graph-analyzer\graph_analyzer.py", line 191, in main
    predicted_class = node_class_predictor(new_graph)
  File "c:\Users\username\Desktop\some-calculator\graph-analyzer\utilities_module.py", line 3379, in node_class_predictor
    x_test, y_test, clf = custom_classifier(emb_df)
  File "c:\Users\username\Desktop\some-calculator\graph-analyzer\utilities_module.py", line 3337, in custom_classifier
    parameter_initializer(x_train, y_train, x_test, y_test)
  File "c:\Users\username\Desktop\some-calculator\graph-analyzer\utilities_module.py", line 3129, in parameter_initializer
    randomized_search(model, distribution, features_train, target_train, features_validation,
  File "c:\Users\username\Desktop\some-calculator\graph-analyzer\utilities_module.py", line 3079, in randomized_search
    search = randomizedsearch_estimator.fit(x_train, y_train)
  File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\sklearn\model_selection\_search.py", line 875, in fit
    self._run_search(evaluate_candidates)
  File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\sklearn\model_selection\_search.py", line 1749, in _run_search   
    evaluate_candidates(
  File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\sklearn\model_selection\_search.py", line 811, in evaluate_candidates
    candidate_params = list(candidate_params)
  File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\sklearn\model_selection\_search.py", line 324, in __iter__       
    params[k] = v.rvs(random_state=rng)
  File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 473, in rvs
    return self.dist.rvs(*self.args, **kwds)
  File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 1068, in rvs
    raise ValueError("Domain error in arguments.")
ValueError: Domain error in arguments.

报错核心是超参数分布违反了SGDClassifier的参数合法范围，同时对scipy的uniform分布参数理解有误：

1. uniform分布参数误用：scipy的uniform(loc, scale)生成的是[loc, loc+scale]区间的随机数，而非[loc, scale]。比如你写的uniform(45000, 55000)会生成45000到100000的数，完全不符合参数要求。
2. epsilon取值严重超标：该参数仅针对huber、epsilon_insensitive等损失函数，合理取值为小正数（如0.01-0.5），你设置的45000-55000完全超出范围。
3. power_t参数非法：该参数仅在learning_rate='invscaling'时生效，要求必须大于0，但你设置的是-100000到-50000的负数，直接违反参数定义域。
4. 损失函数与参数不匹配：部分损失函数（如hinge、log_loss）不需要epsilon参数，随机搜索时抽到这类组合会直接报错。

修正后的参数分布示例 #

def searching_list():
    return [(SGDClassifier(random_state=SEED, learning_rate='optimal', class_weight='balanced'), {
        'alpha': uniform(0.0001, 0.5),  # 正则化系数，合理范围1e-4到0.5
        'l1_ratio': uniform(0, 1),  # elasticnet混合比例，合法范围0到1
        'max_iter': randint(1000, 10000),  # 迭代次数1000-10000足够，无需5万级
        'tol': uniform(1e-6, 1e-3),  # 停止阈值，通常1e-6到1e-3
        'epsilon': uniform(0.01, 0.5),  # 针对huber等损失的合理小正数范围
        'power_t': uniform(0.1, 2),  # 必须大于0，常用范围0.1到2
        'loss': [
            'hinge', 'log_loss', 'modified_huber', 'squared_hinge', 'perceptron',
            'huber', 'epsilon_insensitive', 'squared_epsilon_insensitive'
        ],
        'penalty': ['l2', 'l1', 'elasticnet']
    })]

额外优化建议 #

• 移除squared_error损失函数：该损失是用于回归任务的，不适合分类场景。
• 使用条件参数分布避免无效组合：通过字典列表实现参数与损失函数的绑定，比如：

  'loss': ['hinge', 'log_loss', 'modified_huber', 'squared_hinge', 'perceptron'],
  'power_t': uniform(0.1, 2),
  # 仅针对需要epsilon的损失函数配置参数
  **[{
      'loss': ['huber', 'epsilon_insensitive', 'squared_epsilon_insensitive'],
      'epsilon': uniform(0.01, 0.5)
  }]**
  

内容的提问来源于stack exchange，提问作者user19316680