PyTorch Lightning指标日志异常问题咨询

控制台输出日志：

v_num:z3_3  val_loss:3.105  val_kappa:0.34   val_accuracy:0.295 
            train_loss:2.436    train_kappa: nan  train_accuracy:0.0

train_loss显示正常！

疑问列表 #

• a) 为何训练准确率（train_accuracy）始终为0，而验证准确率却并非如此？
• b) 为何训练Kappa值为NaN，而验证Kappa值正常？
• c) 为何验证指标先显示？按逻辑模型应先训练再评估，训练指标应在上方。

完整代码 #

class Classifier(pl.LightningModule):
  def __init__(self, model_obj):
    super().__init__()
    self.model = model_obj.model
    self.config = model_obj.config
    self.layer_lr = model_obj.layer_lr

    self.kappa = torchmetrics.CohenKappa(task = 'multiclass' , num_classes = self.config['num_classes'], weights = 'quadratic')
    self.accuracy = torchmetrics.Accuracy(task = 'multiclass' , num_classes = self.config['num_classes'])
    self.criterion = torch.nn.CrossEntropyLoss()

  def training_step(self, batch, batch_idx):
    x, y = batch
    y_hat = self.model(x)
    loss = self.criterion(y_hat, y.long())
    self.log("train_loss", loss,on_step = False ,on_epoch=True, prog_bar=True, logger=True)
    self.accuracy(y_hat, y)
    self.kappa(y_hat, y)
    return loss
  
  def validation_step(self, batch, batch_idx):
    x, y = batch
    y_hat = self.model(x)
    loss = self.criterion(y_hat, y.long())
    self.log("val_loss", loss, on_epoch=True, prog_bar=True, logger=True)
    self.accuracy(y_hat, y)
    self.kappa(y_hat, y)
    return loss
  
  def test_step(self, batch, batch_idx):
    x, y = batch
    y_hat = self.model(x)
    loss = self.criterion(y_hat, y.long())
    self.log("test_loss", loss, on_epoch=True, prog_bar=True, logger=True)
    self.accuracy(y_hat, y)
    self.kappa(y_hat, y)
    return loss
  
  def on_train_epoch_end(self):
    
    self.log("train_kappa", self.kappa,on_step=False,  on_epoch=True, prog_bar=True, logger=True)
    self.log("train_accuracy", self.accuracy, on_epoch=True,prog_bar=True, logger=True)

  def on_validation_epoch_end(self):
    
    self.log("val_kappa", self.kappa,on_step = False, on_epoch=True, prog_bar=True, logger=True)
    self.log("val_accuracy", self.accuracy, on_epoch=True,prog_bar=True, logger=True)

  def on_test_epoch_end(self):
    
    self.log("test_kappa", self.kappa,on_step = False,  on_epoch=True, prog_bar=True, logger=True)
    self.log("test_accuracy", self.accuracy, on_epoch=True,prog_bar=True, logger=True)

  
  def configure_optimizers(self):
    optim =  torch.optim.Adam(self.layer_lr, lr = self.config['lr'])   # https://pytorch.org/docs/stable/optim.html
    lr_scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optim, patience=3, factor=0.5, threshold=0.001, cooldown =2,verbose=True)
    return [optim], [{'scheduler': lr_scheduler, 'interval': 'epoch', 'monitor': 'train_loss', 'name': 'lr_scheduler'}]

a) 训练准确率始终为0的原因 #

训练、验证阶段共用了同一个self.accuracy指标实例，状态未按阶段重置：

1. 训练阶段先累计准确率数据
2. 验证阶段继续往同一个实例累加验证数据
3. 验证结束调用log时，会自动计算并重置指标状态
4. 进入on_train_epoch_end时，指标实例已经被清空，此时log的是空状态下的准确率，结果为0。

b) 训练Kappa值为NaN的原因 #

与准确率问题逻辑一致：
共用的self.kappa实例在验证阶段结束后被重置，训练阶段的累计数据完全丢失。空状态下计算Kappa时，因没有有效样本或类别分布异常（如仅单一类别），最终输出NaN。

c) 验证指标先显示的原因 #

PyTorch Lightning默认执行顺序为：训练epoch结束 → 运行验证阶段 → 运行on_train_epoch_end。因此验证阶段的指标会先被输出到控制台，之后才会输出训练阶段指标，导致验证指标显示在上方。

为训练、验证、测试阶段分别创建独立的指标实例，彻底避免状态污染，修改后的核心代码如下：

class Classifier(pl.LightningModule):
  def __init__(self, model_obj):
    super().__init__()
    self.model = model_obj.model
    self.config = model_obj.config
    self.layer_lr = model_obj.layer_lr

    # 为不同阶段创建独立指标实例
    self.train_kappa = torchmetrics.CohenKappa(task='multiclass', num_classes=self.config['num_classes'], weights='quadratic')
    self.train_accuracy = torchmetrics.Accuracy(task='multiclass', num_classes=self.config['num_classes'])
    
    self.val_kappa = torchmetrics.CohenKappa(task='multiclass', num_classes=self.config['num_classes'], weights='quadratic')
    self.val_accuracy = torchmetrics.Accuracy(task='multiclass', num_classes=self.config['num_classes'])
    
    self.test_kappa = torchmetrics.CohenKappa(task='multiclass', num_classes=self.config['num_classes'], weights='quadratic')
    self.test_accuracy = torchmetrics.Accuracy(task='multiclass', num_classes=self.config['num_classes'])
    
    self.criterion = torch.nn.CrossEntropyLoss()

  def training_step(self, batch, batch_idx):
    x, y = batch
    y_hat = self.model(x)
    loss = self.criterion(y_hat, y.long())
    self.log("train_loss", loss, on_step=False, on_epoch=True, prog_bar=True, logger=True)
    # 使用训练专属指标
    self.train_accuracy(y_hat, y)
    self.train_kappa(y_hat, y)
    return loss
  
  def validation_step(self, batch, batch_idx):
    x, y = batch
    y_hat = self.model(x)
    loss = self.criterion(y_hat, y.long())
    self.log("val_loss", loss, on_epoch=True, prog_bar=True, logger=True)
    # 使用验证专属指标
    self.val_accuracy(y_hat, y)
    self.val_kappa(y_hat, y)
    return loss
  
  def on_train_epoch_end(self):
    self.log("train_kappa", self.train_kappa, on_step=False, on_epoch=True, prog_bar=True, logger=True)
    self.log("train_accuracy", self.train_accuracy, on_epoch=True, prog_bar=True, logger=True)

  def on_validation_epoch_end(self):
    self.log("val_kappa", self.val_kappa, on_step=False, on_epoch=True, prog_bar=True, logger=True)
    self.log("val_accuracy", self.val_accuracy, on_epoch=True, prog_bar=True, logger=True)

若需调整指标显示顺序，可在log时通过rank参数控制进度条中的显示位置，或自定义进度条格式。

内容的提问来源于stack exchange，提问作者Sarvagya Porwal