如何查看RStudio中文档分类模型测试集的分类详情及对应文件名？

Hey there! Let's work through how to pull those detailed classification results for your test dataset—specifically getting the filenames of contracts labeled as "x" by your model. Here's a practical, step-by-step approach tailored to typical document classification workflows like the one Tim DAuria covers:

1. First, Link Predictions to Filenames #

The key here is to make sure you're tracking filenames alongside your test data when you load it, then pairing those filenames with the model's predictions.

If you're using pandas for data handling:

Assuming your test data is stored in a CSV (or you can load it into a DataFrame) with columns for filename and text:

import pandas as pd

# Load your test data (adjust the file path/columns to match your setup)
test_data = pd.read_csv("your_test_contracts.csv")

# Get predictions from your model (replace with your model's predict method)
predictions = model.predict(test_data["text"])

# Add predictions as a new column to your test DataFrame
test_data["predicted_class"] = predictions

If you're using a custom data loader (like PyTorch):

Modify your dataset class to store filenames, then collect them during prediction:

import os
import torch
from torch.utils.data import Dataset, DataLoader

class ContractDataset(Dataset):
    def __init__(self, file_paths, tokenizer):
        self.texts = []
        self.filenames = []
        for path in file_paths:
            with open(path, "r", encoding="utf-8") as f:
                self.texts.append(f.read())
            self.filenames.append(os.path.basename(path))  # Store just the filename
        self.tokenizer = tokenizer

    def __getitem__(self, idx):
        encoded = self.tokenizer(
            self.texts[idx],
            truncation=True,
            padding="max_length",
            return_tensors="pt"
        )
        # Return encoded text + filename
        return {k: v.squeeze(1) for k, v in encoded.items()}, self.filenames[idx]

    def __len__(self):
        return len(self.texts)

# Initialize your test dataset and loader
test_dataset = ContractDataset(your_test_file_paths, your_tokenizer)
test_loader = DataLoader(test_dataset, batch_size=8, shuffle=False)

# Collect predictions and filenames
all_preds = []
all_filenames = []
model.eval()

with torch.no_grad():
    for batch in test_loader:
        inputs, filenames = batch
        outputs = model(**inputs)
        # Convert model outputs to class labels (adjust based on your model's output)
        preds = torch.argmax(outputs.logits, dim=1).tolist()
        all_preds.extend(preds)
        all_filenames.extend(filenames)

# Map numerical predictions back to your class labels (if needed)
label_map = {0: "x", 1: "y", 2: "z"}  # Update with your actual label encoding
predicted_classes = [label_map[pred] for pred in all_preds]

# Create a DataFrame to pair filenames and predictions
test_data = pd.DataFrame({
    "filename": all_filenames,
    "predicted_class": predicted_classes
})

2. Filter for Contracts Labeled "x" #

Once you have your paired filenames and predictions, filtering is straightforward:

# Get all contracts predicted as "x"
class_x_contracts = test_data[test_data["predicted_class"] == "x"]

# Extract just the filenames
x_filenames = class_x_contracts["filename"].tolist()

# Print or save the results
print(f"Found {len(x_filenames)} contracts labeled as 'x':")
for name in x_filenames:
    print(name)

3. Save Results for Future Reference #

If you want to keep a record of all test set predictions, export the full DataFrame to a CSV:

test_data.to_csv("test_classification_results.csv", index=False)

You can open this file in Excel, Google Sheets, or load it back into pandas anytime to check other classes or analyze misclassifications.

Just adjust the code snippets to match your model's specific setup (whether it's scikit-learn, Hugging Face Transformers, etc.)—the core idea is always to track filenames alongside your predictions so you can link them later.

内容的提问来源于stack exchange，提问作者dhowie