MLP在pendigits数据集上完全不学习的问题排查求助

核心问题定位 #

从补充信息可知，网络权重从未更新是训练/验证/测试指标完全恒定的根本原因，以下是关键问题点和修复方案：

1. 学习率调度器调用时机错误 #

训练函数中，scheduler.step()被放在了每个数据split（train/val/test）的循环内部，每个epoch会执行3次学习率调整，大概率导致学习率快速衰减至0，优化器无法更新参数。

修复代码：
将scheduler.step()移到每个epoch的末尾，所有数据split处理完成之后：

def train(net, loaders, optimizer, criterion, epochs=100, dev=dev, save_param = True, model_name="only-pendigits"):
    torch.manual_seed(myseed)
    try:
        net = net.to(dev)
        print(net)
        history_loss = {"train": [], "val": [], "test": []}
        history_accuracy = {"train": [], "val": [], "test": []}
        
        for epoch in range(epochs):
            sum_loss = {"train": 0, "val": 0, "test": 0}
            sum_accuracy = {"train": 0, "val": 0, "test": 0}
            
            # 处理所有split
            for split in ["train", "val", "test"]:
                for (input, labels) in loaders[split]:
                    input = input.to(dev)
                    labels = labels.to(dev)
                    optimizer.zero_grad()
                    pred = net(input)
                    loss = criterion(pred, labels)
                    sum_loss[split] += loss.item()
                    
                    if split == "train":
                        loss.backward()
                        optimizer.step()
                    
                    _,pred_labels = pred.max(1)
                    batch_accuracy = (pred_labels == labels).sum().item()/input.size(0)
                    sum_accuracy[split] += batch_accuracy
            
            # 每个epoch只调整一次学习率
            scheduler.step()
            
            # 后续指标计算、打印逻辑不变
            epoch_loss = {split: sum_loss[split]/len(loaders[split]) for split in ["train", "val", "test"]}
            epoch_accuracy = {split: sum_accuracy[split]/len(loaders[split]) for split in ["train", "val", "test"]}
            for split in ["train", "val", "test"]:
                history_loss[split].append(epoch_loss[split])
                history_accuracy[split].append(epoch_accuracy[split])
            print(f"Epoch {epoch+1}:",
                  f"TrL={epoch_loss['train']:.4f},",
                  f"TrA={epoch_accuracy['train']:.4f},",
                  f"VL={epoch_loss['val']:.4f},",
                  f"VA={epoch_accuracy['val']:.4f},",
                  f"TeL={epoch_loss['test']:.4f},",
                  f"TeA={epoch_accuracy['test']:.4f},",
                  f"LR={optimizer.param_groups[0]['lr']:.5f},")
    except KeyboardInterrupt:
        print("Interrupted")
    finally:
        # 绘图逻辑不变
        plt.title("Loss")
        for split in ["train", "val", "test"]:
            plt.plot(history_loss[split], label=split)
        plt.legend()
        plt.show()
        plt.title("Accuracy")
        for split in ["train", "val", "test"]:
            plt.plot(history_accuracy[split], label=split)
        plt.legend()
        plt.show()

2. 标签类型不匹配 #

CrossEntropyLoss要求标签必须是torch.long类型，但从DataFrame中取出的标签是Python int，转成张量后默认是torch.int32或torch.float32，会导致损失计算异常，梯度为0。

修复方案：
在数据加载的__getitem__方法中，将标签转为torch.LongTensor：

def __getitem__(self, idx):
    if torch.is_tensor(idx):
        idx = idx.tolist()

    tabular = self.tabular.iloc[idx, 0:]
    y = tabular["class"]
    # 转换标签类型为long
    y = torch.LongTensor([y])[0]
    
    tabular = tabular[['input1', 'input2', 'input3', 'input4', 'input5', 'input6', 'input7',
       'input8', 'input9', 'input10', 'input11', 'input12', 'input13',
       'input14', 'input15', 'input16']]
    tabular = tabular.tolist()
    tabular = torch.FloatTensor(tabular)
    
    if self.transform:
        tabular = self.transform(tabular)

    return tabular, y

3. 输入特征未归一化 #

Pendigits的输入特征范围是0-100，未归一化会导致网络激活值饱和（比如ReLU在大输入下持续激活，梯度消失），或者损失值过大导致优化器无法正常更新参数。

修复方案：
将输入特征归一化到0-1范围，或者标准化到均值0、方差1：

# 方案1：归一化到0-1
def __getitem__(self, idx):
    # 其他逻辑不变
    tabular = tabular.tolist()
    tabular = torch.FloatTensor(tabular) / 100.0  # 除以最大值100
    # 后续逻辑不变

# 方案2：标准化（更推荐）
# 先计算训练集的均值和方差
train_features = pentrain[['input1', 'input2', 'input3', 'input4', 'input5', 'input6', 'input7',
       'input8', 'input9', 'input10', 'input11', 'input12', 'input13',
       'input14', 'input15', 'input16']].values
mean = train_features.mean(axis=0)
std = train_features.std(axis=0)

# 在Dataset中应用
def __getitem__(self, idx):
    # 其他逻辑不变
    tabular = tabular.tolist()
    tabular = (torch.FloatTensor(tabular) - mean) / std
    # 后续逻辑不变

4. 优化器初始化验证 #

确认优化器初始化时正确传入了模型参数，比如：

optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
# 或者SGD
optimizer = torch.optim.SGD(model.parameters(), lr=0.01, momentum=0.9)

如果未传入model.parameters()，优化器没有可更新的参数，权重自然不会变化。

验证步骤 #

1. 优先修复学习率调度器的调用位置，这是最可能直接导致权重不更新的原因
2. 验证标签类型是否正确
3. 添加输入归一化
4. 检查优化器初始化

按上述步骤修改后，重新训练应该能看到权重更新，指标开始变化。

内容的提问来源于stack exchange，提问作者CasellaJr