使用Transformers库计算NER指标时触发ValueError（元组为空）

在基于Hugging Face Transformers库开发命名实体识别（NER）任务时，自定义compute_metrics函数处理EvalPrediction对象时触发ValueError: Tuple predictions is empty.错误，核心问题出在EvalPrediction对象的结构解析和变量引用上。

相关代码 #

from transformers.trainer_utils import EvalPrediction


def compute_metrics(eval_preds):
    """
    Compute evaluation metrics for Named Entity Recognition (NER) tasks.

    Parameters:
    eval_preds (EvalPrediction): An object containing the predicted logits and the true labels.

    Returns:
    A dictionary containing precision, recall, F1 score, and accuracy.
    """
    if not isinstance(eval_preds, EvalPrediction):
        raise ValueError("Invalid eval_preds structure. Expected an EvalPrediction object.")

    predictions = eval_preds.predictions

    if isinstance(predictions, tuple):
        if len(predictions) > 0:
            pred_logits = predictions[0]
        else:
            raise ValueError("Tuple predictions is empty.")
    else:
        pred_logits = predictions

    # Ensure pred_logits has at least two dimensions
    if len(pred_logits.shape) == 1:
        pred_logits = np.expand_dims(pred_logits, axis=0)

    # Get predicted labels by argmax along the token dimension
    pred_labels = np.argmax(pred_logits, axis=2)
    # ... (rest of your code)
      # Filter out padding tokens where label is -100
    predictions = [
        [label_list[pred] for (pred, label) in zip(pred_label, true_label) if label != -100]
        for pred_label, true_label in zip(pred_labels, labels)
    ]

    # Filter out padding tokens in true labels
    true_labels = [
        [label_list[label] for label in true_label if label != -100]
        for true_label in labels
    ]

    # Compute metrics
    results = metric.compute(predictions=predictions, references=true_labels)

    return {
        "precision": results["overall_precision"],
        "recall": results["overall_recall"],
        "f1": results["overall_f1"],
        "accuracy": results["overall_accuracy"],
    }

trainer = Trainer(
    model,
    args,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["validation"],
    data_collator=data_collator,
    tokenizer=tokenizer,
    compute_metrics=compute_metrics
)

trainer.train()

触发的错误信息 #

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-103-3435b262f1ae> in <cell line: 1>()
----> 1 trainer.train()

5 frames
<ipython-input-101-8c3cb1696dcb> in compute_metrics(eval_preds)
     21             pred_logits = predictions[0]
     22         else:
---&gt; 23             raise ValueError("Tuple predictions is empty.")
     24     else:
     25         pred_logits = predictions

ValueError: Tuple predictions is empty.

问题分析 #

1. EvalPrediction.predictions为空元组：通常是模型forward方法返回的输出结构不符合预期，或Trainer处理预测时出现异常，导致未正确生成logits。
2. 未正确获取真实标签：原代码直接使用未定义的labels变量，未从eval_preds.label_ids中提取真实标签，会导致后续过滤逻辑报错。
3. 依赖变量缺失：代码中label_list、metric、numpy未显式导入或初始化，属于潜在问题。

解决方案 #

1. 修正compute_metrics核心逻辑 #

补充缺失依赖，调整predictions解析逻辑，同时增加调试输出排查问题：

import numpy as np
from datasets import load_metric
from transformers.trainer_utils import EvalPrediction

# 提前初始化依赖（根据你的NER标签体系调整）
metric = load_metric("seqeval")
label_list = ["O", "B-PER", "I-PER", "B-LOC", "I-LOC", "B-ORG", "I-ORG"]

def compute_metrics(eval_preds):
    """
    Compute evaluation metrics for Named Entity Recognition (NER) tasks.

    Parameters:
    eval_preds (EvalPrediction): An object containing the predicted logits and the true labels.

    Returns:
    A dictionary containing precision, recall, F1 score, and accuracy.
    """
    if not isinstance(eval_preds, EvalPrediction):
        raise ValueError("Invalid eval_preds structure. Expected an EvalPrediction object.")

    predictions = eval_preds.predictions
    labels = eval_preds.label_ids  # 正确获取真实标签

    # 调试输出，排查predictions结构异常原因
    print(f"Predictions type: {type(predictions)}, shape/content: {predictions}")

    # 处理predictions结构
    if isinstance(predictions, tuple):
        pred_logits = predictions[0] if len(predictions) > 0 else None
    else:
        pred_logits = predictions

    if pred_logits is None or pred_logits.size == 0:
        raise ValueError("No valid logits found in predictions.")

    # 确保logits维度符合要求（batch_size, seq_len, num_labels）
    if len(pred_logits.shape) == 1:
        pred_logits = np.expand_dims(pred_logits, axis=0)
    elif len(pred_logits.shape) != 3:
        raise ValueError(f"Expected logits to be 3D, got shape {pred_logits.shape}")

    # 生成预测标签
    pred_labels = np.argmax(pred_logits, axis=2)

    # 过滤padding标签并映射为真实标签名称
    predictions = [
        [label_list[pred] for (pred, label) in zip(pred_label, true_label) if label != -100]
        for pred_label, true_label in zip(pred_labels, labels)
    ]

    true_labels = [
        [label_list[label] for label in true_label if label != -100]
        for true_label in labels
    ]

    # 计算指标
    results = metric.compute(predictions=predictions, references=true_labels)

    return {
        "precision": results["overall_precision"],
        "recall": results["overall_recall"],
        "f1": results["overall_f1"],
        "accuracy": results["overall_accuracy"],
    }

2. 检查模型输出结构 #

确保使用的NER模型（如BERTForTokenClassification）返回符合Trainer预期的结果：Hugging Face预训练NER模型默认返回TokenClassifierOutput对象，Trainer会自动解析其中的logits作为eval_preds.predictions。如果是自定义模型，需确保forward方法返回值包含有效logits。

3. 验证数据集标签处理 #

确认数据集在tokenization阶段，padding对应的标签已被设置为-100，避免后续过滤逻辑失效。

内容的提问来源于stack exchange，提问作者Alwan Rahmana Subian