如何确定ANN的隐藏层数与神经元数量？ML新手技术问询

Great question—this is one of those classic "more art than science" problems in machine learning, but there are definitely systematic, programmable approaches you can use instead of relying on guesswork or manual tuning, especially for large datasets. Here are the most practical methods:

1. Automated Hyperparameter Search (Grid/Random Search) #

Instead of manually testing combinations, let code do the work by searching through a predefined space of hidden layer counts and neuron numbers, using cross-validation to pick the best-performing configuration.

Random Search (Better for Large Spaces) #

Random search is more efficient than grid search for large datasets because it samples randomly from your parameter space instead of iterating every single combination. You can use tools like scikit-learn with Keras/TensorFlow wrappers:

from sklearn.model_selection import RandomizedSearchCV
from tensorflow.keras.wrappers.scikit_learn import KerasClassifier
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

# Define a function to build your ANN with variable layers/neurons
def build_ann(hidden_layers=1, neurons_per_layer=64):
    model = Sequential()
    # Input layer + first hidden layer
    model.add(Dense(neurons_per_layer, activation='relu', input_shape=(X_train.shape[1],)))
    # Add additional hidden layers
    for _ in range(hidden_layers - 1):
        model.add(Dense(neurons_per_layer, activation='relu'))
    # Output layer (adjust based on your task: e.g., softmax for multi-class)
    model.add(Dense(1, activation='sigmoid'))
    model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
    return model

# Wrap the Keras model for scikit-learn compatibility
ann_classifier = KerasClassifier(build_fn=build_ann, epochs=15, batch_size=64, verbose=0)

# Define your parameter search space
param_grid = {
    'hidden_layers': [1, 2, 3, 4],
    'neurons_per_layer': [32, 64, 128, 256, 512]
}

# Run random search with 3-fold cross-validation
random_search = RandomizedSearchCV(estimator=ann_classifier, 
                                   param_distributions=param_grid,
                                   n_iter=15,  # Number of random combinations to test
                                   cv=3, 
                                   verbose=2)
search_results = random_search.fit(X_train, y_train)

# Get the best configuration
print(f"Optimal hidden layers: {search_results.best_params_['hidden_layers']}")
print(f"Optimal neurons per layer: {search_results.best_params_['neurons_per_layer']}")

Grid Search (For Smaller, Targeted Spaces) #

If you have a narrow set of parameters to test (e.g., only 1-2 hidden layers with 64/128 neurons), grid search will test every combination. Use GridSearchCV instead of RandomizedSearchCV in the code above.

2. Bayesian Optimization (Smart, Iterative Search) #

Bayesian optimization goes a step further than random search: it uses past results to guide future searches, focusing on parameter combinations that are likely to improve performance. Libraries like Optuna or BayesianOptimization make this easy to implement.

Here’s a quick Optuna example:

import optuna
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from sklearn.model_selection import train_test_split

def objective(trial):
    # Suggest parameters to test
    hidden_layers = trial.suggest_int('hidden_layers', 1, 4)
    neurons_per_layer = trial.suggest_categorical('neurons_per_layer', [32, 64, 128, 256])
    
    # Build model
    model = Sequential()
    model.add(Dense(neurons_per_layer, activation='relu', input_shape=(X_train.shape[1],)))
    for _ in range(hidden_layers - 1):
        model.add(Dense(neurons_per_layer, activation='relu'))
    model.add(Dense(1, activation='sigmoid'))
    model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
    
    # Train and evaluate
    history = model.fit(X_train, y_train, epochs=10, batch_size=64, validation_split=0.2, verbose=0)
    return history.history['val_accuracy'][-1]

# Run optimization
study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=20)

# Print best params
print(f"Best parameters found: {study.best_params}")

3. Iterative Model Expansion with Early Stopping #

Start with a simple model (e.g., 1 hidden layer with 32 neurons) and gradually add layers/neurons. Use early stopping to halt training when validation performance stops improving, which prevents overfitting even as you make the model larger.

This approach is intuitive and works well if you want to build up to an optimal model incrementally. Combine it with regularization (like Dropout layers or L2 penalty) to keep large models generalizable.

4. Leverage Transfer Learning (For Very Large Datasets) #

If you’re working with a large, complex dataset (e.g., images, text), you don’t need to design an ANN from scratch. Use pre-trained models (like ResNet for images, BERT for text) and fine-tune them on your data. These models already have optimized layer structures learned from massive datasets, so you skip the hassle of designing hidden layers entirely.

Quick Rule of Thumb (As a Starting Point) #

While automated methods are better for large datasets, a simple starting point is:

• Neurons per hidden layer: Between the size of the input layer and output layer (e.g., if input is 100 features and output is 1, try 50-64 neurons)
• Hidden layers: Start with 1-2 layers, then use automated search to see if adding more helps

Remember: There’s no one-size-fits-all answer, but these programmable methods take the guesswork out of the process and scale well for large datasets.

内容的提问来源于stack exchange，提问作者Amit Wagner