PyTorch训练BERT模型报错：'numpy.float64'无'cpu'属性求解决

首次使用PyTorch运行BERT并训练模型，完成第一个Epoch后出现错误：

AttributeError: 'numpy.float64' object has no attribute 'cpu'

相关代码片段：

history = defaultdict(list)
best_accuracy = 0

for epoch in range(EPOCHS):
    
    # Show details 
    print(f"Epoch {epoch + 1}/{EPOCHS}")
    print("-" * 10)
    
    train_acc, train_loss = train_epoch(
        model,
        train_data_loader,
        loss_fn,
        optimizer,
        device,
        scheduler,
        len(df_train)
    )
    
    print(f"Train loss {train_loss} accuracy {train_acc}")
    
    # Get model performance (accuracy and loss)
    val_acc, val_loss = eval_model(
        model,
        val_data_loader,
        loss_fn,
        device,
        len(df_val)
    )
    
    print(f"Val   loss {val_loss} accuracy {val_acc}")
    print()
    
    history['train_acc'].append(train_acc.cpu())
    history['train_loss'].append(train_loss.cpu())
    history['val_acc'].append(val_acc.cpu())
    history['val_loss'].append(val_loss.cpu())
    
    # If we beat prev performance
    if val_acc > best_accuracy:
        torch.save(model.state_dict(), 'best_model_state.bin')
        best_accuracy = val_acc

• 直接删除代码中所有.cpu()调用，修改后对应代码段如下：

history['train_acc'].append(train_acc)
history['train_loss'].append(train_loss)
history['val_acc'].append(val_acc)
history['val_loss'].append(val_loss)

原因分析 #

报错的核心是：train_epoch和eval_model函数返回的train_acc、train_loss、val_acc、val_loss已经是numpy.float64类型的普通数值，而非PyTorch的Tensor对象。.cpu()是PyTorch Tensor独有的方法，numpy数值没有该属性，因此触发错误。

通常这种情况是因为在train_epoch和eval_model内部，已经通过.item()或.numpy()方法将Tensor转换成了普通数值，用于直接打印和记录，所以无需再调用.cpu()。

内容的提问来源于stack exchange，提问作者Malak