PyTorch Lightning中ReduceLROnPlateau异常：学习率下调后回弹至基准值

你遇到的学习率回弹问题，核心原因是两个调度器的执行逻辑冲突：

• warmup用的LambdaLR是step级调度器，每一步都会运行；
• warmup结束后，你的lambda函数返回1，这会让LambdaLR把学习率重置为优化器的初始基准值，直接覆盖了ReduceLROnPlateau下调后的学习率，导致每下调一次就被立刻拉回初始值。

方法一：修改warmup的lambda函数，让warmup结束后保持当前学习率 #

直接调整lambda逻辑，warmup阶段正常提升学习率，之后返回当前学习率与初始值的比值，确保LambdaLR不会修改已调整的学习率：

def configure_optimizers(self):
    initial_lr = self.params['training']['learning_rate']
    optimizer = torch.optim.Adam(self.parameters(), lr=initial_lr)

    warmup_steps = self.params['training']['warmup_epochs'] * self.params['training']['train_steps_per_epoch']
    
    def lambda_warmup(step):
        if step < warmup_steps:
            return min(step / warmup_steps, 1)
        # 后续返回当前lr与初始lr的比值，让LambdaLR计算后保持当前lr不变
        return optimizer.param_groups[0]['lr'] / initial_lr
    
    warmup_scheduler = {
        'scheduler': torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda=lambda_warmup),
        'interval': 'step',
        'name': 'warmup',
    }
    
    reduce_on_plateau_scheduler = {
        'scheduler': torch.optim.lr_scheduler.ReduceLROnPlateau(
            optimizer,
            mode='min',
            factor=0.1,
            patience=10,
            verbose=True
        ),
        'monitor': 'val_loss'
    }
    
    return [optimizer], [warmup_scheduler, reduce_on_plateau_scheduler]

方法二：用SequentialLR组合warmup与恒等调度器 #

通过SequentialLR让warmup只在指定步数内生效，之后自动切换到不修改学习率的调度器，彻底避免干扰：

def configure_optimizers(self):
    initial_lr = self.params['training']['learning_rate']
    optimizer = torch.optim.Adam(self.parameters(), lr=initial_lr)

    warmup_steps = self.params['training']['warmup_epochs'] * self.params['training']['train_steps_per_epoch']
    
    # 仅在warmup阶段生效的调度器
    warmup_scheduler = torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda=lambda step: step/warmup_steps)
    # warmup结束后使用恒等调度器（不改变学习率）
    after_warmup_scheduler = torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda=lambda step: 1)
    
    # 组合调度器：warmup_steps后自动切换
    combined_warmup_scheduler = torch.optim.lr_scheduler.SequentialLR(
        optimizer,
        schedulers=[warmup_scheduler, after_warmup_scheduler],
        milestones=[warmup_steps]
    )
    
    warmup_scheduler_config = {
        'scheduler': combined_warmup_scheduler,
        'interval': 'step',
        'name': 'warmup',
    }
    
    reduce_on_plateau_scheduler = {
        'scheduler': torch.optim.lr_scheduler.ReduceLROnPlateau(
            optimizer,
            mode='min',
            factor=0.1,
            patience=10,
            verbose=True
        ),
        'monitor': 'val_loss'
    }
    
    return [optimizer], [warmup_scheduler_config, reduce_on_plateau_scheduler]

添加LearningRateMonitor回调，直接在训练日志中查看每一步的学习率，避免TensorBoard平滑因子导致的观察误差：

from pytorch_lightning.callbacks import LearningRateMonitor

trainer = Trainer(
    callbacks=[LearningRateMonitor(logging_interval='step')],
    # 你的其他训练配置
)

内容的提问来源于stack exchange，提问作者GZinn