Great question—multi-label classification’s confusion matrix challenges are super common because the label space explodes fast when you have combinations like (a,b) or (d,c). Let’s break down how the industry handles this, plus a Weka-specific example.

一、核心思路：避免全组合矩阵，聚焦实用评估 #

First off, almost no one actually builds a confusion matrix for every possible label combination—when you have N labels, there are 2^N -1 possible combinations, which gets out of hand really fast (10 labels = 1023 combinations, a 1023x1023 matrix is basically unreadable). Instead, the industry uses two practical simplification strategies:

1. 单标签独立评估（最常用） #

Treat each label as a separate binary classification problem. For example, for label a, build a confusion matrix that only tracks whether samples were correctly labeled as a (regardless of other tags). This approach gives you clear, actionable insights into how well your model performs on each individual label—something stakeholders usually care about most.

2. 聚焦关键标签组合 #

If specific label combinations are high-priority (e.g., (a,b) is a critical case for your use case), only build a small confusion matrix for those targeted combinations. Ignore low-frequency or non-important combinations to keep the matrix interpretable.

3. 用聚合指标替代全矩阵 #

If you need an overall performance view, use micro-averaged or macro-averaged metrics (like precision, recall, F1-score) derived from binary confusion matrices for each label. These metrics aggregate performance across all labels without requiring a massive combinatorial matrix.

二、Weka中的代码实现 #

Weka has built-in support for multi-label classification (via the MultiLabelClassifier wrapper), but it doesn’t natively generate multi-label confusion matrices. Here are two practical ways to implement this:

方式1：生成每个单标签的混淆矩阵 #

This code iterates through each label, converts the multi-label dataset into a binary dataset for that label, then generates a standard confusion matrix:

import weka.core.Instances;
import weka.core.converters.ConverterUtils.DataSource;
import weka.classifiers.Evaluation;
import weka.classifiers.meta.MultiLabelClassifier;
import weka.classifiers.trees.J48;
import weka.filters.Filter;
import weka.filters.unsupervised.attribute.Remove;

public class MultiLabelConfusionMatrix {
    public static void main(String[] args) throws Exception {
        // Load your multi-label ARFF dataset
        DataSource source = new DataSource("your_multi_label_data.arff");
        Instances data = source.getDataSet();
        
        // Configure label indices (assuming last 4 attributes are labels: a, b, c, d)
        int numLabels = 4;
        data.setClassIndex(data.numAttributes() - numLabels);

        // Initialize multi-label classifier (using J48 as base model)
        MultiLabelClassifier mlc = new MultiLabelClassifier();
        mlc.setClassifier(new J48());
        mlc.buildClassifier(data);

        // Generate confusion matrix for each label
        for (int i = 0; i < numLabels; i++) {
            int labelIndex = data.numAttributes() - numLabels + i;
            String labelName = data.attribute(labelIndex).name();

            // Filter to keep only the current label as class attribute
            Remove removeFilter = new Remove();
            removeFilter.setAttributeIndices("-" + (labelIndex + 1)); // Remove other labels
            removeFilter.setInputFormat(data);
            Instances binaryData = Filter.useFilter(data, removeFilter);
            binaryData.setClassIndex(binaryData.numAttributes() - 1);

            // Evaluate binary classification for this label
            Evaluation eval = new Evaluation(binaryData);
            eval.evaluateModel(mlc, binaryData);

            // Print confusion matrix
            System.out.println("=== Confusion Matrix for Label: " + labelName + " ===");
            System.out.println(eval.toMatrixString());
            System.out.println();
        }
    }
}

方式2：生成特定标签组合的混淆矩阵 #

If you need to evaluate a specific combination (e.g., (a,b)), you can manually create a binary problem where the class is whether the sample was correctly predicted as that combination:

// Continue from the code above, focusing on (a,b) combination
int aLabelIndex = data.numAttributes() - 4; // Index of label 'a'
int bLabelIndex = data.numAttributes() - 3; // Index of label 'b'

// Calculate TP, TN, FP, FN for the (a,b) combination
int truePositives = 0;
int trueNegatives = 0;
int falsePositives = 0;
int falseNegatives = 0;

double threshold = 0.5; // Adjust based on your model's threshold

for (int i = 0; i < data.numInstances(); i++) {
    // Check true label: is this sample (a,b)?
    boolean trueAB = data.instance(i).value(aLabelIndex) == 1 
                    && data.instance(i).value(bLabelIndex) == 1;
    
    // Get model predictions for a and b
    double[] predictions = mlc.distributionForInstance(data.instance(i));
    boolean predAB = predictions[aLabelIndex] >= threshold 
                    && predictions[bLabelIndex] >= threshold;

    // Update confusion matrix counts
    if (trueAB && predAB) truePositives++;
    else if (!trueAB && !predAB) trueNegatives++;
    else if (!trueAB && predAB) falsePositives++;
    else if (trueAB && !predAB) falseNegatives++;
}

// Print the custom confusion matrix for (a,b)
System.out.println("=== Confusion Matrix for Combination (a,b) ===");
System.out.println("                Predicted No | Predicted Yes");
System.out.printf("Actual No       | %-10d | %d%n", trueNegatives, falsePositives);
System.out.printf("Actual Yes      | %-10d | %d%n", falseNegatives, truePositives);

内容的提问来源于stack exchange，提问作者Stephen