使用GaussianNB遇稀疏矩阵报错，转稠密矩阵内存溢出求解决方案

作为机器学习新手，使用GaussianNB模型训练时执行以下代码：

classifier_np = GaussianNB()
classifier_np.fit(X_train, y_train)

触发TypeError错误，错误栈如下：

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_23/2974247449.py in <module>
      1 classifier_np = GaussianNB()
----> 2 classifier_np.fit(X_train, y_train)

/opt/conda/lib/python3.7/site-packages/sklearn/naive_bayes.py in fit(self, X, y, sample_weight)
    244         y = self._validate_data(y=y)
    245         return self._partial_fit(
--> 246             X, y, np.unique(y), _refit=True, sample_weight=sample_weight
    247         )
    248 

/opt/conda/lib/python3.7/site-packages/sklearn/naive_bayes.py in _partial_fit(self, X, y, classes, _refit, sample_weight)
    400 
    401         first_call = _check_partial_fit_first_call(self, classes)
--> 402         X, y = self._validate_data(X, y, reset=first_call)
    403         if sample_weight is not None:
    404             sample_weight = _check_sample_weight(sample_weight, X)

/opt/conda/lib/python3.7/site-packages/sklearn/base.py in _validate_data(self, X, y, reset, validate_separately, **check_params)
    579                 y = check_array(y, **check_y_params)
    580             else:
--> 581                 X, y = check_X_y(X, y, **check_params)
    582             out = X, y
    583 

/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py in check_X_y(X, y, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, multi_output, ensure_min_samples, ensure_min_features, y_numeric, estimator)
    974         ensure_min_samples=ensure_min_samples,
    975         ensure_min_features=ensure_min_features,
--> 976         estimator=estimator,
    977     )
    978 

/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator)
    724             copy=copy,
    725             force_all_finite=force_all_finite,
--> 726             accept_large_sparse=accept_large_sparse,
    727         )
    728     else:

/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py in _ensure_sparse_format(spmatrix, accept_sparse, dtype, copy, force_all_finite, accept_large_sparse)
    439     if accept_sparse is False:
    440         raise TypeError(
--> 441             "A sparse matrix was passed, but dense "
    442             "data is required. Use X.toarray() to "
    443             "convert to a dense numpy array."

TypeError: A sparse matrix was passed, but dense data is required. Use X.toarray() to convert to a dense numpy array.

按照提示执行X_train.toarray()将csr_sparse矩阵转换为稠密矩阵时，出现内存溢出问题，寻求更好的替代方案。

解决方案 #

1. 换用支持稀疏矩阵的朴素贝叶斯模型 #

Sklearn中的MultinomialNB或BernoulliNB原生支持稀疏输入，无需转换为稠密矩阵，尤其适合文本类高维稀疏数据场景：

from sklearn.naive_bayes import MultinomialNB
# 若数据是二元特征，也可选用BernoulliNB
classifier = MultinomialNB()
classifier.fit(X_train, y_train)

2. 对稀疏矩阵做特征降维 #

如果必须使用GaussianNB，可先通过降维减少特征数量，降低转换为稠密矩阵的内存压力：

• 基于特征选择的降维：用SelectKBest或SelectPercentile保留与目标变量最相关的特征：

from sklearn.feature_selection import SelectKBest, chi2
# 选择Top 1000个最具区分度的特征（可根据内存调整k值）
selector = SelectKBest(chi2, k=1000)
X_train_reduced = selector.fit_transform(X_train, y_train)
# 转换为稠密矩阵后训练GaussianNB
classifier_np = GaussianNB()
classifier_np.fit(X_train_reduced.toarray(), y_train)

• 基于矩阵分解的降维：使用TruncatedSVD（支持稀疏矩阵的降维方法，效果类似PCA）将高维数据映射到低维空间：

from sklearn.decomposition import TruncatedSVD
# 将特征压缩到500维（可根据需求调整）
svd = TruncatedSVD(n_components=500)
X_train_reduced = svd.fit_transform(X_train)
# 直接用降维后的稠密矩阵训练GaussianNB
classifier_np = GaussianNB()
classifier_np.fit(X_train_reduced, y_train)

3. 分批训练（Partial Fit） #

GaussianNB支持partial_fit方法，可将稀疏矩阵分成小批次，逐批转换为稠密矩阵后训练，避免一次性加载全部数据：

from sklearn.naive_bayes import GaussianNB
import numpy as np

classifier_np = GaussianNB()
batch_size = 1000  # 每批处理的样本数，可根据内存调整
n_samples = X_train.shape[0]
# 提前获取所有类别标签
classes = np.unique(y_train)

# 逐批训练
for i in range(0, n_samples, batch_size):
    X_batch = X_train[i:i+batch_size].toarray()
    y_batch = y_train[i:i+batch_size]
    classifier_np.partial_fit(X_batch, y_batch, classes=classes)

内容的提问来源于stack exchange，提问作者Shaikh Saad Ali