为何两次训练相同Keras模型无法得到一致结果？

Great question—let’s break down why you’re seeing such a massive discrepancy between your two model runs, even though your code looks identical.

The Core Issue #

The biggest culprit here is unscaled input features paired with random weight initialization.

The breast cancer dataset from scikit-learn has features with wildly different ranges: some values are single-digit, while others climb into the hundreds. When you skip normalizing these features:

• The loss function’s landscape becomes extremely steep, making it nearly impossible for the RMSprop optimizer to navigate effectively.
• Random weight initialization can sometimes land your model in a "dead end" starting point—either gradients are too small to update weights (vanishing gradients) or so large they cause unstable updates (exploding gradients). In your second run, it looks like the model got stuck predicting the majority class (your ~53.8% accuracy matches the proportion of one class in your training split) and can’t learn anything from the data.

Your first run got lucky with a better initial weight set that let the optimizer make progress, but relying on luck isn’t a reliable strategy for machine learning.

Fixes for Consistent, High-Performing Results #

Here’s what you can do to fix this and make your training reproducible:

1. Standardize your input data
   This is non-negotiable for neural networks when features have mismatched scales. Use scikit-learn’s StandardScaler to normalize data to a mean of 0 and standard deviation of 1:

from sklearn.preprocessing import StandardScaler

# Fit scaler on training data, apply to both train and test sets
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Train models with scaled data
model.fit(X_train_scaled, y_train, epochs=20, batch_size=50)
model2.fit(X_train_scaled, y_train, epochs=20, batch_size=50)

2. Set random seeds for reproducibility
   Neural networks rely on randomness (weight initialization, data shuffling, etc.). To get identical results across runs, set seeds for all relevant libraries:

import random
import numpy as np
import tensorflow as tf
from keras import backend as K

# Fix random seeds
random.seed(42)
np.random.seed(42)
tf.random.set_seed(42)

# Mitigate GPU parallelism randomness (if using TensorFlow backend)
if K.backend() == 'tensorflow':
    K.set_session(tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph(), 
                                       config=tf.compat.v1.ConfigProto(intra_op_parallelism_threads=1, 
                                                                       inter_op_parallelism_threads=1)))

Place these lines before creating your models to ensure consistent weight initialization.

3. Double-check for accidental data modifications
   Make sure you’re not altering X_train or y_train between model runs (e.g., scaling the data once without reloading it for the second model). Your current split ([:340]/[340:]) is fixed, so that’s not the issue here—but it’s always good to verify.

Why This Works #

Standardizing data flattens the loss function landscape, making it easier for the optimizer to find a path to lower loss. Setting random seeds ensures both models start with the exact same initial weights, so you’ll get identical training results (we’ve also mitigated GPU-related randomness with the session config).

Give these steps a try—you’ll see both models converge to similar high accuracy and low loss values, just like your first successful run.

内容的提问来源于stack exchange，提问作者Franco