如何在Keras训练阶段约束神经网络输出非负且支持为0？

Hey there! Let's tackle your problem: you want your Keras model to produce non-negative outputs (including 0) during training, not as a post-processing step. Your current model uses tanh as the final activation, which outputs values between [-1, 1]—that's why you're getting negative values. Here are the most straightforward and effective ways to adjust your model:

1. Replace the Final Activation Function #

The simplest fix is to swap out the tanh activation with one that naturally produces non-negative outputs. Two great options are:

Option A: ReLU Activation #

ReLU (Rectified Linear Unit) outputs 0 for any negative input and passes positive inputs through unchanged. This directly enforces non-negative outputs:

from keras.models import Sequential
from keras.layers import Dense, Activation

model = Sequential([
    Dense(100, input_shape=(52,)),
    Activation('relu'),
    Dense(40),
    Activation('softmax'),
    Dense(1),
    Activation('relu')  # Replaced tanh with relu
])

model.compile(optimizer='sgd', loss='mean_absolute_error')
model.fit(train_x2, train_y, epochs=200, batch_size=52)

Pros: Fast computation, widely used, perfect for hard non-negative constraints.
Cons: Can lead to "dead neurons" if inputs are consistently negative (though less likely here since it's the final layer).

Option B: Softplus Activation #

Softplus is a smooth, differentiable alternative to ReLU. It outputs values in (0, ∞) and has no hard cutoff, which can lead to more stable gradients during training:

# Same model structure, just change the final activation
model = Sequential([
    Dense(100, input_shape=(52,)),
    Activation('relu'),
    Dense(40),
    Activation('softmax'),
    Dense(1),
    Activation('softplus')  # Replaced tanh with softplus
])

Pros: Smooth gradients, no dead neurons, ideal if you want continuous non-negative outputs.
Cons: Slightly slower computation than ReLU (negligible for most use cases).

2. Add a Non-Negative Constraint to the Final Layer's Weights (Less Common) #

If you want to keep your activation function (though not recommended here) but still enforce non-negative outputs, you can apply a non-negative constraint to the final dense layer's weights. This ensures the layer's raw output is non-negative before activation:

from keras.constraints import NonNeg

model = Sequential([
    Dense(100, input_shape=(52,)),
    Activation('relu'),
    Dense(40),
    Activation('softmax'),
    Dense(1, kernel_constraint=NonNeg()),  # Add non-negative weight constraint
    Activation('linear')  # Use linear activation since weights are constrained
])

Note: This only works if the input to the final layer is non-negative (which it might be here, since the previous layer uses softmax). If the input has negative values, this won't guarantee non-negative outputs—so activation functions are still the better choice.

3. Custom Loss Function with Penalty for Negative Outputs #

If you need more control (e.g., allow small negative values but penalize them heavily), you can define a custom loss function that adds a penalty term when outputs are negative:

import keras.backend as K

def custom_loss(y_true, y_pred):
    # Base MAE loss
    mae = K.mean(K.abs(y_true - y_pred))
    # Penalty for negative outputs: multiply negative values by a factor (e.g., 10)
    penalty = K.mean(K.maximum(-y_pred, 0) * 10)
    return mae + penalty

# Update your model compilation
model.compile(optimizer='sgd', loss=custom_loss)

Pros: Flexible, lets you tune how harshly negative outputs are penalized.
Cons: Adds complexity, and you might need to adjust the penalty factor to avoid over-penalizing.

For your use case, replacing the final activation with ReLU or Softplus is the cleanest and most effective solution. It directly enforces non-negative outputs during training without extra complexity.

内容的提问来源于stack exchange，提问作者Sus20200