AraBertV2分类任务报错：Dimension specified as 0 but tensor has no dimensions

核心错误原因 #

• 张量维度不匹配：tokenizer.encode_plus设置return_tensors="pt"会生成**(1, max_length)**形状的张量，但torch.tensor(label)是0维标量，导致TensorDataset初始化时，各张量第一维度（样本数）不一致，触发维度检查错误。
• 数据集构建逻辑错误：当前把每个单独样本包装成一个TensorDataset，再用random_split拆分，不符合PyTorch Dataset的使用规范，DataLoader无法正确批量加载这类嵌套的数据集。

修复后的完整代码 #

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification, AdamW
from torch.utils.data import TensorDataset, DataLoader, RandomSampler, SequentialSampler
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
import pandas as pd

model_name = "aubmindlab/bert-base-arabertv2"
# 预训练tokenizer自带正确词汇表，无需手动指定vocab_size
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

max_length = 943

# 批量预处理所有文本，避免单个样本的维度问题
def preprocess_batch(texts, labels):
    tokens = tokenizer(
        texts.tolist(),
        max_length=max_length,
        padding="max_length",
        truncation=True,
        return_attention_mask=True,
        return_tensors="pt"
    )
    # 将标签转为一维张量，与input_ids维度匹配
    labels_tensor = torch.tensor(labels, dtype=torch.long)
    # 用所有样本的张量构建单个TensorDataset
    dataset = TensorDataset(
        tokens["input_ids"],
        tokens["attention_mask"],
        labels_tensor
    )
    return dataset

X = df["fully_cleaned_text_no_stopwords"].values
y = df["target"].values

# 构建完整数据集后再拆分
full_dataset = preprocess_batch(X, y)

train_size = int(0.6 * len(full_dataset))
test_size = len(full_dataset) - train_size
train_dataset, test_dataset = torch.utils.data.random_split(full_dataset, [train_size, test_size])

train_dataloader = DataLoader(
    train_dataset,
    sampler=RandomSampler(train_dataset),
    batch_size=32
)
test_dataloader = DataLoader(
    test_dataset,
    sampler=SequentialSampler(test_dataset),
    batch_size=32
)

optimizer = AdamW(model.parameters(), lr=5e-5)

model.to(device)
model.train()
for epoch in range(25):
    total_loss = 0
    for batch in train_dataloader:
        input_ids = batch[0].to(device)
        attention_mask = batch[1].to(device)
        labels = batch[2].to(device)
        
        model.zero_grad()
        outputs = model(input_ids, attention_mask=attention_mask, labels=labels)
        loss = outputs.loss
        total_loss += loss.item()
        
        loss.backward()
        optimizer.step()
    
    print(f"Epoch {epoch+1}, Average Loss: {total_loss/len(train_dataloader):.4f}")

model.eval()
y_true = []
y_pred = []
for batch in test_dataloader:
    input_ids = batch[0].to(device)
    attention_mask = batch[1].to(device)
    labels = batch[2].to(device)
    
    with torch.no_grad():
        outputs = model(input_ids, attention_mask=attention_mask)
    
    logits = outputs.logits
    predictions = torch.argmax(logits, dim=1)
    y_true.extend(labels.tolist())
    y_pred.extend(predictions.tolist())

accuracy = accuracy_score(y_true, y_pred)
precision = precision_score(y_true, y_pred)
recall = recall_score(y_true, y_pred)
f1 = f1_score(y_true, y_pred)

print("Accuracy:", accuracy)
print("Precision:", precision)
print("Recall:", recall)
print("F1 score:", f1)

关键修复点说明 #

• 移除手动指定的vocab_size：预训练tokenizer的词汇表大小固定，手动设置会破坏原有词汇表，导致tokenize错误。
• 批量预处理文本：直接对所有文本调用tokenizer，避免单个样本处理的维度问题，同时提升效率。
• 统一张量维度：将标签转换为(n_samples,)的一维张量，与input_ids、attention_mask的第一维度保持一致。
• 修正数据集构建逻辑：用所有样本的张量构建单个TensorDataset后再拆分，符合PyTorch数据加载规范。
• 优化训练循环：将optimizer.zero_grad()移到batch循环开头，避免梯度累积错误，并添加loss监控。

内容的提问来源于stack exchange，提问作者Hero of time