调用tensor.to(torch.float)后仍遇PyTorch数据与权重类型不匹配问题求解

环境信息 #

• OS：Ubuntu 16.04
• Python版本：3.8
• PyTorch版本：1.10.1

问题详情 #

训练InceptionNetModel时触发类型不匹配错误，报错内容：

RuntimeError: Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.FloatTensor) should be the same

错误出现在卷积层前向传播环节，相关代码片段：

# Instantiate the neural network and optimizer
net = InceptionNetModel()
net.to(device)
optimizer = optim.Adam(net.parameters(), lr=0.01)
criterion = nn.BCELoss()

# Train the neural network
for epoch in range(500):
    net.train()
    iteration = 0
    for batch_idx, (apc_batch, pump_batch, vent_batch, kpc_batch, dms_batch, info_batch, y_batch) in enumerate(
            train_tabular_dataloader):
        apc_batch, pump_batch, vent_batch, kpc_batch, dms_batch, info_batch, label_batch = \
            apc_batch.to(device, torch.float64), pump_batch.to(device, torch.float64), vent_batch.to(device, torch.float64), kpc_batch.to(device, torch.float64), dms_batch.to(device, torch.float64), info_batch.to(device, torch.float64), y_batch.to(device, torch.float64)

        optimizer.zero_grad()
        y_pred = net(apc_batch, pump_batch, vent_batch, kpc_batch, dms_batch, info_batch)

错误堆栈：

File "/home/user/anaconda3/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 446, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/user/anaconda3/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 442, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.FloatTensor) should be the same

解决方法 #

方案1：统一输入为FloatTensor（推荐） #

PyTorch默认初始化的模型权重是torch.float32（FloatTensor），但代码中把所有输入张量转成了torch.float64（DoubleTensor），导致类型不兼容。修改输入转换代码，将torch.float64替换为torch.float或torch.float32：

apc_batch, pump_batch, vent_batch, kpc_batch, dms_batch, info_batch, label_batch = \
    apc_batch.to(device, torch.float), pump_batch.to(device, torch.float), vent_batch.to(device, torch.float), kpc_batch.to(device, torch.float), dms_batch.to(device, torch.float), info_batch.to(device, torch.float), y_batch.to(device, torch.float)

方案2：将模型转为DoubleTensor适配输入 #

如果业务确实需要使用DoubleTensor精度训练，可以在模型部署到设备后，将模型整体转为double类型：

net = InceptionNetModel()
net.to(device)
net = net.double()  # 或 net.to(torch.double)
optimizer = optim.Adam(net.parameters(), lr=0.01)

额外注意事项 #

• 计算损失时，确保标签张量label_batch的类型与模型输出y_pred一致，避免新的类型错误。
• 大部分PyTorch预定义模型默认使用FloatTensor，该类型计算速度更快、显存占用更低，无特殊高精度需求时优先选用。

内容的提问来源于stack exchange，提问作者jabberwoo