将Keras代码转换为PyTorch后分类模型准确率下降问题排查

Hey there! Let's dig into why your PyTorch model isn't matching the Keras performance—there are a few key implementation differences that are almost certainly causing the accuracy drop and sky-high loss. Let's break them down one by one:

1. Mismatched Fully Connected Layer Dimension #

In your Keras model, the first dense layer is Dense(512), but your PyTorch code uses nn.Linear(10368, 1024). This changes the model's feature capacity drastically and breaks the learned representation flow from the convolutional layers.

Fix: Update the PyTorch linear layer to match Keras's dimension:

self.fc1 = nn.Linear(10368, 512)
self.batch_3 = nn.BatchNorm1d(512)

2. Softmax Layer Conflict with Loss Function #

Keras pairs a final softmax activation with categorical_crossentropy loss, but PyTorch's nn.CrossEntropyLoss already includes LogSoftmax and NLLLoss logic under the hood. Adding an extra nn.Softmax() in your model leads to incorrect loss calculations—this is the most likely culprit behind your loss jumping from 0.2 to 4.5.

Fix: Remove the Softmax_1 layer entirely. Your forward pass should end with:

y = self.fc2(y)
return y

Ensure you're using nn.CrossEntropyLoss() as your training loss function (it handles both one-hot and integer labels, just like Keras's categorical/sparse categorical crossentropy).

3. Input Channel Order Misalignment #

Keras uses NHWC format (batch, height, width, channels) for inputs, while PyTorch uses NCHW (batch, channels, height, width). If you haven't converted your input data from Keras's format to PyTorch's, your convolutional layers are processing dimensions incorrectly, which would ruin model performance.

Fix: Permute input dimensions before passing them to the model:

# If your input is shaped (batch, 48, 48, 3)
x = x.permute(0, 3, 1, 2)  # Reshapes to (batch, 3, 48, 48)

4. Batch Normalization Momentum Difference #

Keras's BatchNormalization uses a default momentum of 0.99, while PyTorch's nn.BatchNorm2d/nn.BatchNorm1d defaults to 0.1. This changes how running mean/variance are updated during training, affecting convergence speed and final performance.

Fix: Set momentum to match Keras when defining BatchNorm layers:

self.batch_1 = nn.BatchNorm2d(32, momentum=0.99)
self.batch_2 = nn.BatchNorm2d(128, momentum=0.99)
self.batch_3 = nn.BatchNorm1d(512, momentum=0.99)

Corrected PyTorch Model Code #

Putting all these fixes together, here's the revised model:

import torch.nn as nn

class convnet(nn.Module):
    def __init__(self, num_class):
        super(convnet, self).__init__()
        self.Conv2d_1 = nn.Conv2d(3, 16, (3,3))
        self.relu_1 = nn.ReLU()
        self.Conv2d_2 = nn.Conv2d(16, 32, (3,3))
        self.relu_2 = nn.ReLU()
        self.maxpool_1 = nn.MaxPool2d(kernel_size=2, stride=2)
        self.batch_1 = nn.BatchNorm2d(32, momentum=0.99)
        self.Conv2d_3 = nn.Conv2d(32, 64, (3,3))
        self.relu_3 = nn.ReLU()
        self.Conv2d_4 = nn.Conv2d(64, 128, (3,3))
        self.relu_4 = nn.ReLU()
        self.maxpool_2 = nn.MaxPool2d(kernel_size=2, stride=2)
        self.batch_2 = nn.BatchNorm2d(128, momentum=0.99)
        self.fc1 = nn.Linear(10368, 512)
        self.relu_5 = nn.ReLU()
        self.batch_3 = nn.BatchNorm1d(512, momentum=0.99)
        self.dropout_1 = nn.Dropout(p=0.5)
        self.fc2 = nn.Linear(512, num_class)

    def forward(self, x):
        y = self.Conv2d_1(x)
        y = self.relu_1(y)
        y = self.Conv2d_2(y)
        y = self.relu_2(y)
        y = self.maxpool_1(y)
        y = self.batch_1(y)
        y = self.Conv2d_3(y)
        y = self.relu_3(y)
        y = self.Conv2d_4(y)
        y = self.relu_4(y)
        y = self.maxpool_2(y)
        y = self.batch_2(y)
        y = y.view(y.size(0), -1)
        y = self.fc1(y)
        y = self.relu_5(y)
        y = self.batch_3(y)
        y = self.dropout_1(y)
        y = self.fc2(y)
        return y

Additional Sanity Checks #

• Optimizer Settings: Make sure you're using the same optimizer (e.g., Adam) with identical learning rate, weight decay, and other hyperparameters as your Keras setup.
• Data Preprocessing: Confirm data normalization (mean/std scaling) is exactly the same between frameworks—even small differences here can shift performance.

After applying these fixes, your PyTorch model should perform nearly identically to the Keras version. Let me know if you still see discrepancies!

内容的提问来源于stack exchange，提问作者Hamidreza74