使用DistilBERT微调SST2情感分类时遇IndexError问题求助

使用Hugging Face库微调distilbert-base-uncased模型做SST2数据集的情感分析分类任务时，运行报错：

IndexError: Target -1 is out of bounds.

加载分词后数据集的相关代码如下：

small_train_dataset = encoded_dataset["train"].shuffle(seed=42).select(range(1000))
small_eval_dataset = encoded_dataset["test"].shuffle(seed=42).select(range(1000))
full_train_dataset = encoded_dataset["train"]
full_eval_dataset = encoded_dataset["test"]

训练参数定义：

# Define the training parameters

metric_name = "accuracy"
model_name = model_checkpoint.split("/")[-1]
weight_decay = 0.01
lr = 2e-5
batch_size = 16
num_train_epochs = 5

评估函数：

import numpy as np

# Define a function to evaluate the model

def compute_metrics(eval_pred):
    logits, labels = eval_pred
    predictions = np.argmax(logits, axis=-1)
    return metric.compute(predictions=predictions, references=labels)

训练配置与执行：

# Fine-tune the model

args = TrainingArguments(
    f"{model_name}-finetuned-{task}",
    evaluation_strategy = "steps",             
    eval_steps=10,
    save_strategy = "steps",                   
    learning_rate = lr,                   
    per_device_train_batch_size = batch_size,     
    per_device_eval_batch_size = batch_size,      
    num_train_epochs = num_train_epochs,                
    weight_decay = weight_decay,                    
    load_best_model_at_end = True,           
    metric_for_best_model = metric_name,           
    push_to_hub = False,                    
)

trainer = Trainer(
    model=model,
    args=args,
    train_dataset = small_train_dataset,           
    eval_dataset = small_eval_dataset,            
    tokenizer = tokenizer,               
    compute_metrics = compute_metrics         
)

trainer.train()

已尝试更换评估指标，问题未解决。

这个错误的核心是数据集中存在标签为-1的样本，而模型的分类头针对SST2二分类（标签0/1）设计，-1超出了模型可处理的标签范围。以下是具体排查和解决步骤：

• 排查无效标签样本：先确认数据集中是否存在标签为-1的样本，执行以下代码：

  # 查看训练集标签的唯一值
  print(small_train_dataset['label'].unique())
  # 统计标签为-1的样本数量
  print(sum(1 for label in small_train_dataset['label'] if label == -1))
  

• 过滤无效标签样本：如果确实存在标签为-1的样本，直接过滤掉这些无效数据：

  small_train_dataset = small_train_dataset.filter(lambda x: x['label'] != -1)
  small_eval_dataset = small_eval_dataset.filter(lambda x: x['label'] != -1)
  

• 确认模型分类头维度：初始化模型时必须指定num_labels=2，确保分类头与二分类任务匹配：

  from transformers import DistilBertForSequenceClassification
  
  model = DistilBertForSequenceClassification.from_pretrained(model_checkpoint, num_labels=2)
  

• 检查数据处理逻辑：如果使用了自定义data_collator，确认是否在批量处理时错误修改了标签值，建议使用默认的DataCollatorWithPadding：

  from transformers import DataCollatorWithPadding
  
  data_collator = DataCollatorWithPadding(tokenizer=tokenizer)
  # 初始化Trainer时传入data_collator
  trainer = Trainer(
      model=model,
      args=args,
      train_dataset=small_train_dataset,
      eval_dataset=small_eval_dataset,
      tokenizer=tokenizer,
      compute_metrics=compute_metrics,
      data_collator=data_collator
  )
  

内容的提问来源于stack exchange，提问作者Navid Aslankhani