TensorFlow实现线性回归均方误差偏高，请求代码问题排查

Hey Tony, let's break down why your model's MSE is sitting around 6 when it should nail this simple 2*x + y linear task—even with correct matrix dimensions, there are several common pitfalls that often trip up new TensorFlow users here.

Here are the most likely fixes to try:

1. Normalize Your Input Features #

This is probably the biggest culprit. If your x and y input values have a large range (e.g., 0-100 instead of 0-1), the network's initial weights can't scale properly, leading to unstable gradient updates and slow convergence. TensorFlow makes this easy with built-in normalization layers:

# Add this before your model layers
normalizer = tf.keras.layers.Normalization(axis=-1)
normalizer.adapt(your_input_data)  # Replace with your actual input array

Feed the normalized inputs into your network, and you'll likely see a huge drop in MSE.

2. Drop Nonlinear Activations #

If your network uses activation functions like ReLU, you're forcing a nonlinear model to fit a perfectly linear function. That's like using a hammer to screw in a nail—unnecessary and inefficient. For linear regression, use linear activation (which is the default if you omit the activation parameter in Dense layers):

# Replace any activated Dense layers with this
tf.keras.layers.Dense(units=1, activation=None)

A single linear layer is all you need to model 2*x + y—no hidden layers required (though adding hidden linear layers won't hurt, it's overkill).

3. Tweak Training Hyperparameters #

• Learning Rate: If your optimizer's learning rate is too small, the model takes forever to converge; too large, and it bounces around the optimal weights without settling. For Adam, try learning_rate=0.01 or 0.001 (the default is 0.001, but sometimes a slightly higher rate works better for simple tasks).
• Training Epochs: You might not be training long enough. Try doubling or tripling your epoch count, or add early stopping to stop once validation loss stops improving:

  early_stopping = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)
  model.fit(..., callbacks=[early_stopping])
  

• Batch Size: If your batch size is too large, gradient updates are less frequent; too small, and they're noisy. Stick to a middle ground like 32 or 64 for small datasets.

4. Check Weight Initialization #

Default weight initialization (Glorot uniform) works for most cases, but for simple linear tasks, smaller initial weights can help the model converge faster. Try explicitly setting a smaller standard deviation:

tf.keras.layers.Dense(units=1, activation=None,
                      kernel_initializer=tf.keras.initializers.RandomNormal(stddev=0.01))

Example Fixed Code #

Here's a trimmed-down version of your model that should get near-zero MSE:

import tensorflow as tf
import numpy as np

# Generate sample data (adjust ranges as needed)
np.random.seed(42)
x = np.random.rand(1000) * 10
y = np.random.rand(1000) * 10
true_outputs = 2 * x + y

# Normalize inputs
normalizer = tf.keras.layers.Normalization(axis=-1)
normalizer.adapt(np.column_stack((x, y)))

# Build minimal linear model
model = tf.keras.Sequential([
    normalizer,
    tf.keras.layers.Dense(1, activation=None)
])

# Compile with adjusted learning rate
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.01),
              loss='mean_squared_error')

# Train with validation and early stopping
history = model.fit(
    np.column_stack((x, y)), true_outputs,
    epochs=100,
    batch_size=32,
    validation_split=0.2,
    callbacks=[tf.keras.callbacks.EarlyStopping(patience=5)],
    verbose=1
)

# Evaluate
final_mse = model.evaluate(np.column_stack((x, y)), true_outputs, verbose=0)
print(f"Final MSE: {final_mse:.4f}")

Give these changes a shot—you should see your MSE drop to nearly zero, which is what you'd expect for such a simple linear task.

内容的提问来源于stack exchange，提问作者Tony