如何为HuggingFace零样本分类Transformer模型的预测结果提取SHAP值？

Great question! Let's walk through actionable improvements to get clear, useful SHAP text explanations for your zero-shot classification workflow:

1. Target Specific Labels in Your SHAP Calculations #

Zero-shot classification relies on explicit target labels, but your current setup doesn't pass these labels to the SHAP explainer—this can lead to ambiguous or irrelevant explanations. Instead, wrap your pipeline in a custom function that binds the target labels, so SHAP calculates token contributions for exactly the labels you care about.

2. Optimize the SHAP Explainer for Zero-Shot Pipelines #

The default shap.Explainer works with pipelines, but for zero-shot tasks, it's better to explicitly define a prediction function that outputs probabilities for your target labels. This ensures SHAP correctly maps token impacts to each label's score.

3. Clean Up Special Tokens in Visualizations #

Models like BART add special tokens (e.g., <s>, </s>) during tokenization, which clutter the SHAP text plot. You can filter these out before visualization to make explanations more readable.

4. Handle Multi-Label Explanations Explicitly #

If you're using multi_label=True, generate separate SHAP explanations for each label instead of a combined view. This makes it easier to see which tokens drive each individual label's prediction.

Modified Code Example #

Here's how to implement these improvements:

import shap
from transformers import AutoModelForSequenceClassification, AutoTokenizer, ZeroShotClassificationPipeline

# Initialize model and pipeline
model = AutoModelForSequenceClassification.from_pretrained('facebook/bart-large-mnli')
tokenizer = AutoTokenizer.from_pretrained('facebook/bart-large-mnli')
pipe = ZeroShotClassificationPipeline(model=model, tokenizer=tokenizer, return_all_scores=True)

def score_and_visualize(text, target_labels):
    # First, run the prediction to confirm outputs
    prediction = pipe(text, target_labels, multi_label=True)
    print("Prediction Results:", prediction[0])
    
    # Define a custom prediction function for SHAP
    def predict_fn(texts):
        # Return probabilities for each target label
        outputs = pipe(texts, target_labels, multi_label=True)
        return [ [score['score'] for score in output] for output in outputs ]
    
    # Initialize SHAP explainer with the custom function and tokenizer
    explainer = shap.Explainer(predict_fn, tokenizer)
    
    # Calculate SHAP values
    shap_values = explainer([text])
    
    # Filter out special tokens for cleaner visualization
    clean_tokens = [token for token in shap_values.data[0] if not token.startswith("<") and not token.endswith(">")]
    clean_shap_values = shap_values.values[0][[i for i, token in enumerate(shap_values.data[0]) if not token.startswith("<") and not token.endswith(">")]]
    
    # Visualize for each label separately
    for idx, label in enumerate(target_labels):
        print(f"\n--- SHAP Explanation for Label: {label} ---")
        shap.plots.text(shap.Explanation(values=clean_shap_values[:, idx], data=clean_tokens, out_names=[label]))

# Test with your example
example_text = "This is an example text about snowflakes in the summer"
labels = ["weather", "sports", "computer industry"]
score_and_visualize(example_text, labels)

Key Notes: #

• The custom predict_fn ensures SHAP focuses only on your target labels, avoiding unnecessary computations.
• Filtering special tokens removes noise from the visualization, making it easier to identify impactful words like "snowflakes" and "summer" for the "weather" label.
• Generating per-label explanations lets you clearly see how each token contributes to individual label scores.

内容的提问来源于stack exchange，提问作者Pete