Keras框架下CNN模型精度低，如何实现验证准确率停滞时衰减SGD学习率？

Got it, let's work through this problem step by step. When your validation accuracy hits a plateau and stops improving, adjusting the SGD learning rate automatically is a great way to nudge your model toward better performance. Here's exactly how to implement this in Keras, with both built-in and custom options:

Step 1: Use Keras' Built-in ReduceLROnPlateau Callback #

The easiest and most reliable method is leveraging Keras' pre-built callback—it’s designed specifically for this scenario. Let’s break it down:

Import the Callback #

First, pull in the callback from Keras:

from keras.callbacks import ReduceLROnPlateau

Configure the Learning Rate Reducer #

Create an instance of the callback with parameters tailored to your training setup:

lr_reducer = ReduceLROnPlateau(
    monitor='val_acc',        # Track validation accuracy to trigger decay
    factor=0.1,               # Reduce learning rate to 10% of its current value when triggered
    patience=5,               # Wait 5 epochs with no val_acc improvement before decaying
    min_lr=1e-7,              # Don't let the learning rate drop below this minimum
    verbose=1                 # Print a message when the learning rate is adjusted
)

Quick note: If you’re using a newer version of Keras (or TensorFlow Keras), the metric name might be val_accuracy instead of val_acc—double-check your model’s compiled metrics to match the correct label.

Set Up SGD Optimizer #

Define your SGD optimizer with your preferred initial learning rate and momentum:

from keras.optimizers import SGD

sgd_optimizer = SGD(learning_rate=0.01, momentum=0.9)  # Tune initial lr/momentum to your task

Compile and Train with the Callback #

Compile your model as usual, then pass the lr_reducer to the callbacks argument in model.fit():

# Compile the model
model.compile(
    optimizer=sgd_optimizer,
    loss='categorical_crossentropy',  # Adjust loss function to match your task
    metrics=['acc']                   # Use 'accuracy' if needed for newer Keras versions
)

# Start training with automatic learning rate decay
training_history = model.fit(
    x_train, y_train,
    epochs=100,
    batch_size=32,
    validation_data=(x_val, y_val),
    callbacks=[lr_reducer]
)

What This Does Behind the Scenes #

• After each epoch, the callback checks if val_acc has improved over the best value seen so far.
• If no improvement occurs for patience consecutive epochs, it multiplies the current learning rate by factor.
• It will never drop below min_lr to avoid the learning rate becoming too small to make meaningful updates.

Bonus: Custom Callback (For Full Control) #

If you need more tailored logic (like different decay rules or additional actions), you can build a custom callback:

from keras.callbacks import Callback
import keras.backend as K

class CustomLRDecay(Callback):
    def __init__(self, patience=5, factor=0.1, min_lr=1e-7):
        super().__init__()
        self.patience = patience
        self.factor = factor
        self.min_lr = min_lr
        self.best_val_acc = 0.0
        self.wait_count = 0

    def on_epoch_end(self, epoch, logs=None):
        current_val_acc = logs.get('val_acc')
        # Update best accuracy and reset wait counter if we see improvement
        if current_val_acc > self.best_val_acc:
            self.best_val_acc = current_val_acc
            self.wait_count = 0
        else:
            self.wait_count += 1
            # Trigger decay if we've waited long enough
            if self.wait_count >= self.patience:
                current_lr = K.get_value(self.model.optimizer.lr)
                new_lr = max(current_lr * self.factor, self.min_lr)
                K.set_value(self.model.optimizer.lr, new_lr)
                print(f"\nEpoch {epoch+1}: Reducing learning rate to {new_lr:.6f}")
                self.wait_count = 0

# Use the custom callback like this
custom_lr_decay = CustomLRDecay(patience=5, factor=0.1, min_lr=1e-7)
training_history = model.fit(..., callbacks=[custom_lr_decay])

That said, the built-in ReduceLROnPlateau is well-tested and sufficient for most use cases—save the custom callback for when you need unique behavior.

内容的提问来源于stack exchange，提问作者YeongHwa Jin