Keras回归网络输出反缩放层实现及输出缩放必要性咨询

Great question! Let's tackle this from two angles: embedding your custom scaling logic into the Keras model, and whether scaling your outputs is actually necessary.

1. Embedding Scaling/Inverse Scaling into Your Model #

You’re absolutely right that you can bake your (xi - mean)/max(|X|) scaling into the model itself—this way, you don’t have to manually track and load means/max values for inference. Here’s how to do it cleanly:

Step 1: Precompute Your Statistics First #

First, calculate the mean and max absolute value for every input and output feature using your training data (never use test data for this!).

import numpy as np
import tensorflow as tf
from tensorflow.keras import layers, Model

# Assume X_train (shape: [num_samples, 60]) and y_train (shape: [num_samples, 35]) are your training data
input_means = X_train.mean(axis=0)
input_max_abs = np.max(np.abs(X_train), axis=0)
output_means = y_train.mean(axis=0)
output_max_abs = np.max(np.abs(y_train), axis=0)

Step 2: Build Custom Layers (or Use Lambda Layers) #

For robustness, I recommend creating custom non-trainable layers to handle scaling and inverse scaling. This ensures your stats are saved with the model, avoiding any manual tracking:

class InputScaler(layers.Layer):
    def __init__(self, means, max_abs, **kwargs):
        super().__init__(**kwargs)
        # Store stats as non-trainable weights
        self.means = self.add_weight(
            name="input_means",
            shape=means.shape,
            initializer=tf.keras.initializers.Constant(means),
            trainable=False
        )
        self.max_abs = self.add_weight(
            name="input_max_abs",
            shape=max_abs.shape,
            initializer=tf.keras.initializers.Constant(max_abs),
            trainable=False
        )
    
    def call(self, inputs):
        return (inputs - self.means) / self.max_abs

class OutputInverseScaler(layers.Layer):
    def __init__(self, means, max_abs, **kwargs):
        super().__init__(**kwargs)
        self.means = self.add_weight(
            name="output_means",
            shape=means.shape,
            initializer=tf.keras.initializers.Constant(means),
            trainable=False
        )
        self.max_abs = self.add_weight(
            name="output_max_abs",
            shape=max_abs.shape,
            initializer=tf.keras.initializers.Constant(max_abs),
            trainable=False
        )
    
    def call(self, inputs):
        return inputs * self.max_abs + self.means

Step 3: Create Two Models (Training vs. Inference) #

You’ll want two slightly different models: one for training (which outputs scaled values to match your scaled targets) and one for inference (which outputs raw, un-scaled predictions):

# Core network architecture (shared between training and inference)
def build_core_network():
    inputs = layers.Input(shape=(60,))
    scaled_inputs = InputScaler(input_means, input_max_abs)(inputs)
    
    # Add your regression layers here—adjust based on your needs
    x = layers.Dense(128, activation="relu")(scaled_inputs)
    x = layers.Dense(64, activation="relu")(x)
    scaled_outputs = layers.Dense(35)(x)
    
    return inputs, scaled_outputs

# Training model: outputs scaled values to match scaled y_train
train_inputs, train_outputs = build_core_network()
training_model = Model(inputs=train_inputs, outputs=train_outputs)
training_model.compile(optimizer="adam", loss="mse")

# Prepare scaled training targets
scaled_y_train = (y_train - output_means) / output_max_abs

# Train the model
training_model.fit(X_train, scaled_y_train, epochs=50, batch_size=32, validation_split=0.1)

# Inference model: adds inverse scaling to return raw predictions
inference_inputs, scaled_inference_outputs = build_core_network()
raw_outputs = OutputInverseScaler(output_means, output_max_abs)(scaled_inference_outputs)
inference_model = Model(inputs=inference_inputs, outputs=raw_outputs)

# Save the inference model—now you can load it later and use raw inputs directly!
inference_model.save("my_scaled_regression_model.h5")

# Example inference (no need to load means/max_abs manually!)
loaded_model = tf.keras.models.load_model("my_scaled_regression_model.h5")
raw_predictions = loaded_model.predict(X_test)  # Returns values in original scale

If you prefer a quicker approach (without custom layers), you can use Lambda layers instead—just note that the stats are hardcoded into the layer, so you need to ensure they’re correctly captured when saving the model. Custom layers are more explicit and less error-prone, though.

2. Should You Skip Scaling Outputs? #

Short answer: No, don’t skip scaling your outputs—here’s why:

• Your output variables have wildly different scales (±10 vs. 1e5). If you train on raw targets, your loss function (like MSE) will be completely dominated by the large-scale variables. The model will prioritize minimizing error for the 1e5-scale variables, and the small-scale variables will get almost no attention, leading to terrible predictions for them.
• Scaling ensures all output variables contribute equally to the loss, so the model learns to predict every variable well.

The only exception would be if you use a weighted loss function, where you assign higher weights to small-scale variables to balance their contribution. But this requires tuning weights manually, which is more work than just scaling your outputs. Scaling is a simpler, more reliable solution for your use case.

内容的提问来源于stack exchange，提问作者Jenna