如何基于csr_matrix稀疏矩阵计算置换重要性？

问题背景 #

使用scikit-learn的permutation_importance函数处理csr_matrix类型的稀疏矩阵时，会抛出错误：TypeError: A sparse matrix was passed, but dense data is required.。由于稀疏矩阵的密集版本无法放入内存，需要寻找无需转密集的解决方案。

复现代码 #

import pandas as pd

data = {
   'y': [1, 0, 0, 1, 1, 1],
  'categ': ['dog', 'cat', 'dog', 'ant', 'fox', 'seal'],
  'size': ['big', 'small', 'big', 'tiny', 'medium', 'big']
}
df = pd.DataFrame(data)

X = df[['categ', 'size']]
y = df['y']

from sklearn.preprocessing import OneHotEncoder
encoder = OneHotEncoder(sparse_output=True)
X = encoder.fit_transform(X)

from sklearn.linear_model import LogisticRegression
logit = LogisticRegression()
logit.fit(X, y)

# 此步骤抛出错误
from sklearn.inspection import permutation_importance
perm = permutation_importance(logit, X, y)

解决方案 #

以下两种方法无需将稀疏矩阵转为密集格式即可完成计算：

方法一：基于原始分类特征置换（推荐） #

由于稀疏矩阵由分类特征one-hot编码生成，直接置换one-hot列无实际解释性，应在原始特征层面执行置换，再重新编码为稀疏矩阵：

import pandas as pd
import numpy as np
from sklearn.preprocessing import OneHotEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# 原始数据
data = {
   'y': [1, 0, 0, 1, 1, 1],
  'categ': ['dog', 'cat', 'dog', 'ant', 'fox', 'seal'],
  'size': ['big', 'small', 'big', 'tiny', 'medium', 'big']
}
df = pd.DataFrame(data)
X_raw = df[['categ', 'size']]
y = df['y']

# 编码并训练模型
encoder = OneHotEncoder(sparse_output=True)
X_encoded = encoder.fit_transform(X_raw)
logit = LogisticRegression()
logit.fit(X_encoded, y)

# 计算基准性能分数
base_score = accuracy_score(y, logit.predict(X_encoded))

# 逐个计算原始特征的置换重要性
perm_results = []
for feature in X_raw.columns:
    # 复制原始数据并置换当前特征
    X_permuted_raw = X_raw.copy()
    X_permuted_raw[feature] = np.random.permutation(X_permuted_raw[feature])
    # 重新编码为稀疏矩阵
    X_permuted_encoded = encoder.transform(X_permuted_raw)
    # 计算置换后的模型性能
    permuted_score = accuracy_score(y, logit.predict(X_permuted_encoded))
    # 重要性为基准分数与置换后分数的差值
    importance = base_score - permuted_score
    perm_results.append({'feature': feature, 'permutation_importance': importance})

# 输出结果
print(pd.DataFrame(perm_results))

该方法的优势：

• 内存占用低，无需处理高维稀疏矩阵
• 置换逻辑贴合分类特征的实际意义，结果更具解释性

方法二：自定义稀疏矩阵列置换逻辑 #

若需针对one-hot后的稀疏矩阵列计算重要性，可通过自定义函数利用稀疏矩阵的切片操作实现置换，避免转密集：

import numpy as np
from sklearn.metrics import accuracy_score

def sparse_permutation_importance(model, X_sparse, y, n_repeats=5):
    n_features = X_sparse.shape[1]
    importances = np.zeros((n_repeats, n_features))
    base_score = accuracy_score(y, model.predict(X_sparse))
    
    for repeat in range(n_repeats):
        for feature_idx in range(n_features):
            # 复制稀疏矩阵
            X_permuted = X_sparse.copy()
            # 获取当前列的非零元素索引
            non_zero_rows = X_permuted[:, feature_idx].nonzero()[0]
            # 置换非零元素的顺序
            permuted_values = X_permuted[non_zero_rows, feature_idx].toarray().flatten()
            np.random.shuffle(permuted_values)
            X_permuted[non_zero_rows, feature_idx] = permuted_values
            # 计算置换后的性能分数
            permuted_score = accuracy_score(y, model.predict(X_permuted))
            importances[repeat, feature_idx] = base_score - permuted_score
    
    # 返回平均重要性和标准差
    return {
        'importances_mean': np.mean(importances, axis=0),
        'importances_std': np.std(importances, axis=0)
    }

# 使用自定义函数计算
perm = sparse_permutation_importance(logit, X, y)
print("平均重要性：", perm['importances_mean'])
print("重要性标准差：", perm['importances_std'])

注意：该方法针对one-hot列单独置换，结果的解释性较弱，仅适用于需细粒度分析one-hot特征的场景。

内容的提问来源于stack exchange，提问作者Dudelstein