Python与R计算稳健协方差矩阵的结果差异求助

我正在以R代码为参考开发Python统计包，发现两者计算稳健协方差矩阵时结果存在显著差异。使用完全相同的输入数据x，Python通过MinCovDet().fit(x)得到的协方差矩阵x_cov，与R通过covMcd(x)得到的结果数值差异极大，已排除算法随机性影响，现寻求差异原因及解决建议。

Python代码与结果 #

代码 #

import numpy as np
from sklearn.covariance import MinCovDet

# 测试数据
x = np.array([
    [0.5488135, 0.71518937, 0.60276338, 0.54488318],
    [0.4236548, 0.64589411, 0.43758721, 0.891773],
    [0.96366276, 0.38344152, 0.79172504, 0.52889492],
    [0.56804456, 0.92559664, 0.07103606, 0.0871293],
    [0.0202184, 0.83261985, 0.77815675, 0.87001215],
    [0.97861834, 0.79915856, 0.46147936, 0.78052918],
    [0.11827443, 0.63992102, 0.14335329, 0.94466892],
    [0.52184832, 0.41466194, 0.26455561, 0.77423369],
    [0.45615033, 0.56843395, 0.0187898, 0.6176355],
    [0.61209572, 0.616934, 0.94374808, 0.6818203],
    [0.3595079, 0.43703195, 0.6976312, 0.06022547],
    [0.66676672, 0.67063787, 0.21038256, 0.1289263],
    [0.31542835, 0.36371077, 0.57019677, 0.43860151],
    [0.98837384, 0.10204481, 0.20887676, 0.16130952],
    [0.65310833, 0.2532916, 0.46631077, 0.24442559],
    [0.15896958, 0.11037514, 0.65632959, 0.13818295],
    [0.19658236, 0.36872517, 0.82099323, 0.09710128],
    [0.83794491, 0.09609841, 0.97645947, 0.4686512],
    [0.97676109, 0.60484552, 0.73926358, 0.03918779],
    [0.28280696, 0.12019656, 0.2961402, 0.11872772]
])

# 拟合MCD模型
mcd = MinCovDet().fit(x)

# 获取稳健协方差矩阵
x_cov = mcd.covariance_

结果 #

x_cov = array([[ 0.06669275,  0.01987514,  0.01294049,  0.0235569 ],
              [ 0.01987514,  0.0421388 , -0.00541365,  0.0462657 ],
              [ 0.01294049, -0.00541365,  0.06601437, -0.02931285],
              [ 0.0235569 ,  0.0462657 , -0.02931285,  0.08961389]])

R代码与结果 #

代码 #

# 测试数据
x <- matrix(c(
  0.5488135, 0.71518937, 0.60276338, 0.54488318,
  0.4236548, 0.64589411, 0.43758721, 0.891773,
  0.96366276, 0.38344152, 0.79172504, 0.52889492,
  0.56804456, 0.92559664, 0.07103606, 0.0871293,
  0.0202184, 0.83261985, 0.77815675, 0.87001215,
  0.97861834, 0.79915856, 0.46147936, 0.78052918,
  0.11827443, 0.63992102, 0.14335329, 0.94466892,
  0.52184832, 0.41466194, 0.26455561, 0.77423369,
  0.45615033, 0.56843395, 0.0187898, 0.6176355,
  0.61209572, 0.616934, 0.94374808, 0.6818203,
  0.3595079, 0.43703195, 0.6976312, 0.06022547,
  0.66676672, 0.67063787, 0.21038256, 0.1289263,
  0.31542835, 0.36371077, 0.57019677, 0.43860151,
  0.98837384, 0.10204481, 0.20887676, 0.16130952,
  0.65310833, 0.2532916, 0.46631077, 0.24442559,
  0.15896958, 0.11037514, 0.65632959, 0.13818295,
  0.19658236, 0.36872517, 0.82099323, 0.09710128,
  0.83794491, 0.09609841, 0.97645947, 0.4686512,
  0.97676109, 0.60484552, 0.73926358, 0.03918779,
  0.28280696, 0.12019656, 0.2961402, 0.11872772
), nrow = 20, ncol = 4, byrow = TRUE)

# 拟合MCD模型
x.mcd <- covMcd(x)

# 获取稳健协方差矩阵
x_cov <- x.mcd$cov

结果 #

x_cov =             [,1]       [,2]        [,3]        [,4]
          [1,]  0.15762177 0.01044705 -0.04043184 -0.01187968
          [2,]  0.01044705 0.09957141  0.03036312  0.08703770
          [3,] -0.04043184 0.03036312  0.06045952 -0.01781794
          [4,] -0.01187968 0.08703770 -0.01781794  0.16634435

差异原因分析及解决建议 #

核心原因 #

1. 缩放因子差异：

   • R的covMcd默认返回一致性校正后的协方差，会对MCD子集的原始协方差进行缩放，使其符合总体协方差的无偏估计；而sklearn的MinCovDet默认返回未做一致性校正的原始MCD协方差。
   • 验证：R中x.mcd$raw.cov是未缩放的原始结果，与Python输出更接近；Python中通过mcd.covariance_ * mcd.scale_可得到与R对齐的一致性校正结果。

2. 初始子集选择逻辑：
   两者的初始子集采样、候选筛选细节（如采样次数、最优子集判定规则）存在差异，即使固定随机种子，也可能选中不同的MCD样本子集，进而影响协方差计算结果。

解决建议 #

1. 统一缩放规则：

   • Python端：计算mcd.covariance_ * mcd.scale_得到一致性校正后的协方差，匹配R的covMcd(x)$cov。
   • R端：若需匹配Python原始结果，提取x.mcd$raw.cov即可。

2. 固定随机种子：

   • Python中设置MinCovDet(random_state=42).fit(x)固定随机种子；R中通过set.seed(42)控制随机数生成，确保子集选择的可复现性。

3. 核对选中子集：
   分别输出两者选中的样本索引（Python：mcd.support_，R：x.mcd$best），确认是否为相同子集。若子集不同，可调整n_subsamples等参数对齐算法逻辑。

内容的提问来源于stack exchange，提问作者Zack Eriksen