在Scikit-learn中如何为自定义类传入任意数量的模型？

Awesome question! This pattern is super useful for building modular, sklearn-compatible estimators, and you absolutely can extend it to support any number of nested models. Let me walk you through how to do this cleanly:

1. Basic Multi-Model Class (Explicitly Named Models) #

The simplest approach is to define each model as a separate parameter in your __init__ method. Since your class inherits from BaseEstimator, sklearn will automatically detect nested estimator parameters and prefix them with the attribute name followed by __.

Here's a working example:

from sklearn.base import BaseEstimator, ClassifierMixin
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import SVC

class MultiModelEnsemble(BaseEstimator, ClassifierMixin):
    def __init__(self,
                 classifier_a=LogisticRegression(),
                 classifier_b=RandomForestClassifier(),
                 classifier_c=SVC(probability=True)):
        # Store each model as an instance attribute
        self.classifier_a = classifier_a
        self.classifier_b = classifier_b
        self.classifier_c = classifier_c

    def fit(self, X, y):
        # Fit all models to the training data
        self.classifier_a.fit(X, y)
        self.classifier_b.fit(X, y)
        self.classifier_c.fit(X, y)
        return self

    def predict(self, X):
        # Example ensemble logic: majority vote
        preds_a = self.classifier_a.predict(X)
        preds_b = self.classifier_b.predict(X)
        preds_c = self.classifier_c.predict(X)
        
        # Count votes for each sample
        vote_counts = preds_a + preds_b + preds_c
        return (vote_counts >= 2).astype(int)

2. Verify Nested Parameters #

When you create an instance and call get_params(), you'll see all nested parameters with the correct prefixes:

ensemble = MultiModelEnsemble()
print(ensemble.get_params().keys())
# Output includes: 'classifier_a__C', 'classifier_b__n_estimators', 'classifier_c__kernel', etc.

3. Integrate with GridSearchCV #

This works seamlessly with GridSearchCV—just specify parameters using the attribute__param syntax:

from sklearn.model_selection import GridSearchCV

param_grid = {
    'classifier_a__C': [0.01, 0.1, 1, 10],
    'classifier_b__n_estimators': [50, 100, 200],
    'classifier_b__max_depth': [3, 5, None],
    'classifier_c__kernel': ['linear', 'rbf']
}

grid_search = GridSearchCV(ensemble, param_grid, cv=5, scoring='accuracy')
grid_search.fit(X_train, y_train)

print(grid_search.best_params_)
# Will show the optimal combination of nested parameters

4. Flexible Approach (Arbitrary Number of Models) #

If you want to support a dynamic number of models (without hardcoding attributes), use a dictionary to store models. sklearn will still parse nested parameters correctly, using the dictionary keys as part of the parameter prefix:

from sklearn.ensemble import GradientBoostingClassifier

class FlexibleEnsemble(BaseEstimator, ClassifierMixin):
    def __init__(self, models=None):
        # Default to an empty dict if no models are provided
        self.models = models or {}

    def fit(self, X, y):
        for model in self.models.values():
            model.fit(X, y)
        return self

    def predict_proba(self, X):
        # Example: average predicted probabilities
        probas = [model.predict_proba(X)[:, 1] for model in self.models.values()]
        return sum(probas) / len(probas)

    def predict(self, X):
        return (self.predict_proba(X) >= 0.5).astype(int)

Use it like this:

flex_ensemble = FlexibleEnsemble(
    models={
        'logreg': LogisticRegression(),
        'rf': RandomForestClassifier(),
        'gbm': GradientBoostingClassifier()
    }
)

# Access nested parameters
print(flex_ensemble.get_params()['models__logreg__C'])

# GridSearchCV with dynamic models
flex_param_grid = {
    'models__logreg__C': [0.1, 1, 10],
    'models__rf__max_depth': [3, 7, None],
    'models__gbm__learning_rate': [0.01, 0.1]
}

Key Takeaways #

• BaseEstimator does the heavy lifting: Any instance attribute that is itself a sklearn estimator (inherits from BaseEstimator) will have its parameters automatically expanded with the attribute__param syntax.
• Works for any number of models: Whether you hardcode 2 models or pass a dict of 10, sklearn's parameter handling system will adapt.
• Keep logic flexible: Adjust the fit/predict methods to match your use case (ensemble voting, stacking, weighted averaging, etc.)

内容的提问来源于stack exchange，提问作者dre_ml