如何多次训练同一模型？基于sklearn MLP回归器的小数据集训练方案

Got it, let's break down exactly how to implement your plan using scikit-learn's Multi-layer Perceptron Regressor. Your approach of splitting your 100-sample training set 4 times into 75 training / 25 validation subsets is a smart move for small datasets—it helps you gauge how stable your model performs across different data splits.

1. Import Required Libraries #

First, grab all the tools you'll need:

import numpy as np
from sklearn.neural_network import MLPRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error  # Or your preferred regression metric
from sklearn.preprocessing import StandardScaler  # Critical for MLP performance!

2. Prepare & Preprocess Your Dataset #

Assume you have your feature matrix X (shape: (100, num_features)) and target variable y (shape: (100,)). MLPs are sensitive to feature scales, so don't skip preprocessing:

# Scale features to standard normal distribution (mean=0, std=1)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

3. Run 4 Split-Train-Evaluate Cycles #

We'll loop 4 times, each time splitting the data differently, training an MLP, and tracking performance:

# Define MLP hyperparameters (tune these based on your data!)
mlp_config = {
    'hidden_layer_sizes': (32, 16),  # Start small for a 100-sample dataset
    'max_iter': 1500,
    'random_state': 42,  # Ensures consistent model initialization
    'early_stopping': True,  # Prevents overfitting by stopping early if validation stalls
    'alpha': 0.001,  # L2 regularization to combat overfitting
    'validation_fraction': 0.1  # Separate small validation set for early stopping
}

# Store results across splits
validation_mse = []
trained_models = []

# Run 4 unique splits
for split_num in range(4):
    # Split scaled data: 75% train, 25% validation
    X_train, X_val, y_train, y_val = train_test_split(
        X_scaled, y, test_size=0.25, random_state=split_num
    )
    
    # Initialize and train the MLP
    mlp = MLPRegressor(**mlp_config)
    mlp.fit(X_train, y_train)
    
    # Evaluate on validation set
    y_pred = mlp.predict(X_val)
    mse = mean_squared_error(y_val, y_pred)
    validation_mse.append(mse)
    trained_models.append(mlp)
    
    print(f"Split {split_num + 1}: Validation MSE = {mse:.4f}")

# Summarize overall performance
print("\n--- Training Summary ---")
print(f"Average Validation MSE across 4 splits: {np.mean(validation_mse):.4f}")
print(f"Individual MSE scores: {[round(score, 4) for score in validation_mse]}")

Key Tips for Success #

• Feature Scaling: This is non-negotiable for MLPs—without it, features with larger ranges will dominate the model's learning. Always fit the scaler only on training data to avoid data leakage.
• Hyperparameter Tuning: If validation scores vary wildly or are consistently poor, adjust your MLP settings:
  • Reduce hidden_layer_sizes if you suspect overfitting
  • Increase alpha to add more regularization
  • Try different activation functions (e.g., tanh instead of the default relu)
• Model Stability: Large variance in validation scores means your small dataset is driving inconsistent model performance. Consider collecting more data, or switch to a simpler model like a Ridge Regressor if needed.
• Reproducibility: Using split_num as the random_state ensures each split is unique but reproducible. Omit this parameter if you want fully random splits every run.

内容的提问来源于stack exchange，提问作者emax