PyTorch神经网络中int64与Long类型混淆引发前向传播错误排查

问题 #

训练PyTorch神经网络时，前向传播阶段抛出错误：RuntimeError: expected scalar type Long but found Float。已尝试将输入数据转换为Long类型（自定义数据集类中用dtype=torch.long，也试过type(torch.LongTensor)），前向传播前打印输入数据类型为torch.int64（与Long等价），但错误仍未解决。

自定义数据集类代码 #

class MnistTrainDataset(Dataset):
    def __init__(self, df):
        self.X = torch.tensor(df.iloc[:, 1:].values, dtype=torch.long).reshape((-1, 1, 28, 28))
        self.y = torch.tensor(df.iloc[:, 0].values, dtype=torch.long).reshape((-1, 1))
        print(self.X[0])
    
    def __len__(self):
        return len(self.y)
    
    def __getitem__(self, idx):
        return self.X[idx], self.y[idx]

前向传播代码 #

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        print(x.dtype)
        x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
        print(x.dtype)
        x = F.max_pool2d(F.relu(self.conv2(x)), 2)
        print(x.dtype)
        x = torch.flatten(x, 1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

完整错误栈 #

2 for epoch in range(num_epochs):
      3     for X, y in train_iter:
----> 4         y_hat = net(X.type(torch.LongTensor))
      5         l = loss(y_hat.reshape(y.shape), y)
      7         optimiser.zero_grad()

File ~/opt/anaconda3/envs/d2l/lib/python3.10/site-packages/torch/nn/modules/module.py:1130, in Module._call_impl(self, *input, **kwargs)
   1126 # If we don't have any hooks, we want to skip the rest of the logic in
   1127 # this function, and just call forward.
   1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1129         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1130     return forward_call(*input, **kwargs)
   1131 # Do not call functions when jit is used
   1132 full_backward_hooks, non_full_backward_hooks = [], []

/Users/atikshgupta/Desktop/kaggle/digit_mnist/mnist_classifier.ipynb Cell 5' in Net.forward(self, x)
     10 def forward(self, x):
     11     print(x.dtype)
---> 12     x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
     13     print(x.dtype)
     14     x = F.max_pool2d(F.relu(self.conv2(x)), 2)

File ~/opt/anaconda3/envs/d2l/lib/python3.10/site-packages/torch/nn/modules/module.py:1130, in Module._call_impl(self, *input, **kwargs)
   1126 # If we don't have any hooks, we want to skip the rest of the logic in
   1127 # this function, and just call forward.
   1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1129         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1130     return forward_call(*input, **kwargs)
   1131 # Do not call functions when jit is used
   1132 full_backward_hooks, non_full_backward_hooks = [], []

File ~/opt/anaconda3/envs/d2l/lib/python3.10/site-packages/torch/nn/modules/conv.py:457, in Conv2d.forward(self, input)
    456 def forward(self, input: Tensor) -> Tensor:
---> 457     return self._conv_forward(input, self.weight, self.bias)

File ~/opt/anaconda3/envs/d2l/lib/python3.10/site-packages/torch/nn/modules/conv.py:453, in Conv2d._conv_forward(self, input, weight, bias)
    449 if self.padding_mode != 'zeros':
    450     return F.conv2d(F.pad(input, self._reversed_padding_repeated_twice, mode=self.padding_mode),
    451                     weight, bias, self.stride,
    452                     _pair(0), self.dilation, self.groups)
---> 453 return F.conv2d(input, weight, bias, self.stride,
    454                 self.padding, self.dilation, self.groups)

RuntimeError: expected scalar type Long but found Float`

解决方案 #

错误核心是卷积层权重默认是Float类型，但你强制将输入转为Long类型，两者类型不匹配。PyTorch的卷积、全连接等层要求输入为浮点型，Long类型仅适用于分类任务的标签。

修改步骤 #

1. 修正数据集输入类型：将self.X改为torch.float32，标签self.y保持torch.long不变，同时建议将MNIST像素值归一化到0-1区间，加速收敛。
2. 移除训练循环中的多余类型转换：直接传入原始输入即可。

修改后的代码 #

自定义数据集类

class MnistTrainDataset(Dataset):
    def __init__(self, df):
        # 输入转浮点型并归一化
        self.X = torch.tensor(df.iloc[:, 1:].values, dtype=torch.float32).reshape((-1, 1, 28, 28)) / 255.0
        self.y = torch.tensor(df.iloc[:, 0].values, dtype=torch.long).reshape((-1, 1))
    
    def __len__(self):
        return len(self.y)
    
    def __getitem__(self, idx):
        return self.X[idx], self.y[idx]

训练循环修改

for epoch in range(num_epochs):
    for X, y in train_iter:
        # 移除多余的类型转换
        y_hat = net(X)
        l = loss(y_hat.reshape(y.shape), y)
        optimiser.zero_grad()
        l.backward()
        optimiser.step()

补充说明 #

• 卷积、线性层参数默认是torch.float32，输入必须与参数类型一致。
• 标签用torch.long是正确的，交叉熵损失要求目标值为整数类型。

内容的提问来源于stack exchange，提问作者EBT