做多分类时遇TypeError：稀疏矩阵长度模糊，转稠密仍报错求解决

The error you're seeing comes directly from calling len() on sparse matrices (your y_test and y_pred are both sparse). Sparse matrices don't support the len() function because it's ambiguous whether you're asking for the number of rows or the number of non-zero elements. Instead, you need to use .shape[0] to get the number of samples, or handle the sparse matrices appropriately for operations like concatenation.

Key Issues in Your Code #

1. len(y_pred) and len(y_test): These calls are invalid for sparse matrices.
2. Reshaping Sparse Matrices: Trying to reshape sparse matrices directly with reshape() can lead to unexpected behavior; converting to dense arrays (or using sparse-specific tools) is safer for concatenation.

Corrected Code #

Here's your full code with fixes, plus explanations:

from sklearn.naive_bayes import CategoricalNB
from sklearn.datasets import make_multilabel_classification
from sklearn.model_selection import train_test_split
from skmultilearn.adapt import MLkNN
from sklearn.metrics import accuracy_score
import numpy as np
from scipy.sparse import hstack  # For sparse concatenation (optional)

# Generate multi-label sparse dataset
X, y = make_multilabel_classification(
    sparse=True, 
    n_labels=15, 
    return_indicator='sparse', 
    allow_unlabeled=False
)

# Split into training and test sets
X_train, X_test, y_train, y_test = train_test_split(
    X, y, 
    test_size=0.25, 
    random_state=0
)

# Note: MLkNN can handle sparse X matrices, so converting to dense isn't required
# But if you prefer dense, keep these lines:
X_train = X_train.todense()
X_test = X_test.todense()

# Train the MLkNN classifier
classifier = MLkNN(k=20)
classifier.fit(X_train, y_train)

# Make predictions (y_pred is a sparse matrix)
y_pred = classifier.predict(X_test)

# Calculate accuracy (scikit-learn's accuracy_score supports sparse matrices)
print("Model Accuracy:", accuracy_score(y_test, y_pred))

# Fix the concatenation step (two options below)

# Option 1: Convert to dense arrays for numpy concatenation (easier for small datasets)
y_pred_dense = y_pred.toarray()
y_test_dense = y_test.toarray()
concatenated_dense = np.concatenate([y_pred_dense, y_test_dense], axis=1)
print("Dense Concatenated Shape:", concatenated_dense.shape)

# Option 2: Keep matrices sparse (better for large datasets)
concatenated_sparse = hstack([y_pred, y_test])
print("Sparse Concatenated Shape:", concatenated_sparse.shape)

Explanations #

• Avoid len() on Sparse Matrices: Use y_pred.shape[0] if you need the number of samples (rows) instead of len(y_pred).
• Concatenation Options:
  • Dense: Use .toarray() to convert sparse matrices to numpy arrays, then use np.concatenate as usual. This is straightforward for small to medium datasets.
  • Sparse: Use scipy.sparse.hstack to concatenate sparse matrices without converting to dense, which saves memory for large datasets.
• Accuracy Calculation: accuracy_score from scikit-learn natively supports sparse matrices, so you don't need to convert y_test or y_pred to dense for this step.

If you originally intended to reshape each label vector into a single column (e.g., each row is a sample's full label array), you can modify the dense concatenation like this:

# Reshape each label matrix into a single column of arrays
y_pred_col = y_pred_dense.reshape(-1, 1)
y_test_col = y_test_dense.reshape(-1, 1)
concatenated_cols = np.concatenate([y_pred_col, y_test_col], axis=1)

内容的提问来源于stack exchange，提问作者mtr_007