如何用Python基于核密度计算Jaccard指数特征的KL散度？

我有一组形状为(100, 3)的Jaccard Index数值，其中“3”对应3个模型输入参数。我想基于这些Jaccard指数的采样分布，计算任意两组特征组合之间的KL散度，但不知道如何为每一列的Jaccard Index计算概率分布（尤其是核密度估计），进而完成KL散度计算，急需具体操作指引。

我的Jaccard Index数据 #

import numpy as np

jaccard_data = np.array([[0.12    , 1.      , 0.272727],
       [0.074074, 0.882353, 0.208333],
       [0.16    , 0.933333, 0.25    ],
       [0.192308, 0.888889, 0.148148],
       [0.238095, 0.473684, 0.45    ],
       [0.26087 , 0.933333, 0.26087 ],
       [0.333333, 0.75    , 0.3     ],
       [0.2     , 0.722222, 0.166667],
       [0.318182, 0.823529, 0.25    ],
       [0.076923, 0.631579, 0.217391],
       [0.142857, 0.722222, 0.26087 ],
       [0.115385, 0.555556, 0.125   ],
       [0.136364, 0.875   , 0.3     ],
       [0.166667, 0.722222, 0.318182],
       [0.115385, 0.764706, 0.166667],
       [0.166667, 1.      , 0.130435],
       [0.125   , 1.      , 0.08    ],
       [0.272727, 0.666667, 0.181818],
       [0.222222, 0.571429, 0.142857],
       [0.26087 , 0.5     , 0.304348],
       [0.076923, 1.      , 0.333333],
       [0.470588, 0.8     , 0.25    ],
       [0.125   , 0.611111, 0.285714],
       [0.083333, 0.625   , 0.2     ],
       [0.26087 , 0.866667, 0.16    ],
       [0.181818, 0.555556, 0.25    ],
       [0.304348, 0.9375  , 0.318182],
       [0.166667, 0.928571, 0.388889],
       [0.192308, 0.684211, 0.304348],
       [0.26087 , 0.866667, 0.190476],
       [0.2     , 0.555556, 0.3     ],
       [0.076923, 0.705882, 0.16    ],
       [0.115385, 1.      , 0.444444],
       [0.208333, 0.611111, 0.2     ],
       [0.2     , 0.833333, 0.333333],
       [0.125   , 0.555556, 0.333333],
       [0.166667, 0.681818, 0.103448],
       [0.166667, 1.      , 0.166667],
       [0.173913, 0.764706, 0.153846],
       [0.208333, 0.6     , 0.16    ],
       [0.2     , 0.444444, 0.2     ],
       [0.153846, 0.736842, 0.166667],
       [0.074074, 0.777778, 0.4     ],
       [0.071429, 0.9375  , 0.16    ],
       [0.153846, 0.705882, 0.173913],
       [0.111111, 1.      , 0.130435],
       [0.142857, 0.789474, 0.148148],
       [0.166667, 0.722222, 0.148148],
       [0.142857, 0.888889, 0.16    ],
       [0.285714, 0.526316, 0.125   ],
       [0.103448, 0.6     , 0.107143],
       [0.148148, 0.777778, 0.192308],
       [0.137931, 0.714286, 0.2     ],
       [0.181818, 0.8125  , 0.285714],
       [0.136364, 0.388889, 0.125   ],
       [0.227273, 0.588235, 0.25    ],
       [0.136364, 0.625   , 0.238095],
       [0.157895, 0.5625  , 0.095238],
       [0.3     , 0.473684, 0.181818],
       [0.130435, 1.      , 0.285714],
       [0.318182, 0.75    , 0.388889],
       [0.24    , 0.6     , 0.26087 ],
       [0.173913, 0.55    , 0.26087 ],
       [0.173913, 0.684211, 0.217391],
       [0.111111, 1.      , 0.208333],
       [0.115385, 0.684211, 0.2     ],
       [0.227273, 0.8125  , 0.368421],
       [0.227273, 1.      , 0.130435],
       [0.24    , 0.7     , 0.192308],
       [0.173913, 0.764706, 0.272727],
       [0.304348, 0.8125  , 0.333333],
       [0.291667, 0.666667, 0.347826],
       [0.107143, 0.6     , 0.208333],
       [0.192308, 0.842105, 0.2     ],
       [0.217391, 0.714286, 0.12    ],
       [0.217391, 1.      , 0.076923],
       [0.190476, 0.764706, 0.208333],
       [0.033333, 0.571429, 0.153846],
       [0.208333, 0.941176, 0.272727],
       [0.333333, 0.722222, 0.153846],
       [0.153846, 0.823529, 0.217391],
       [0.136364, 0.8     , 0.529412],
       [0.1     , 0.714286, 0.185185],
       [0.142857, 0.785714, 0.333333],
       [0.173913, 0.526316, 0.25    ],
       [0.208333, 0.388889, 0.25    ],
       [0.16    , 0.6     , 0.16    ],
       [0.272727, 0.764706, 0.238095],
       [0.086957, 0.625   , 0.190476],
       [0.103448, 0.684211, 0.24    ],
       [0.238095, 0.444444, 0.173913],
       [0.35    , 0.875   , 0.227273],
       [0.12    , 0.823529, 0.238095],
       [0.28    , 0.722222, 0.178571],
       [0.208333, 0.944444, 0.185185],
       [0.2     , 0.611111, 0.208333],
       [0.095238, 0.473684, 0.12    ],
       [0.190476, 0.733333, 0.210526],
       [0.192308, 0.75    , 0.217391],
       [0.12    , 0.764706, 0.217391]])

一、用核密度估计（KDE）构建每列的概率分布 #

使用scipy.stats.gaussian_kde可以为每一列数据拟合核密度分布，具体操作如下：

from scipy.stats import gaussian_kde

# 为每一列数据创建KDE模型
kde_models = []
for col in range(jaccard_data.shape[1]):
    data_col = jaccard_data[:, col]
    kde = gaussian_kde(data_col)
    kde_models.append(kde)

这段代码会生成3个KDE模型，分别对应3列Jaccard指数的概率分布。调用kde.evaluate(x)就能得到在x点处的概率密度值。

二、计算KL散度 #

KL散度用于衡量两个概率分布的差异，公式为：
$D_{KL}(P||Q) = \int P(x) \log\left(\frac{P(x)}{Q(x)}\right) dx$

由于我们的分布是连续的，需要用数值积分来近似计算：

import numpy as np
from scipy.integrate import quad

def kl_divergence(kde_p, kde_q, x_min, x_max):
    # 定义被积函数
    def integrand(x):
        p = kde_p.evaluate(x)[0]
        q = kde_q.evaluate(x)[0]
        # 避免除以0或取log(0)，添加极小值
        if p <= 1e-10 or q <= 1e-10:
            return 0.0
        return p * np.log(p / q)
    
    # 数值积分计算KL散度
    result, _ = quad(integrand, x_min, x_max)
    return result

# 确定所有Jaccard指数的取值范围，作为积分区间
all_data = jaccard_data.flatten()
x_min = all_data.min() - 0.01  # 扩展一点边界
x_max = all_data.max() + 0.01

# 计算所有特征组合的KL散度
kl_matrix = np.zeros((3, 3))
for i in range(3):
    for j in range(3):
        kl_matrix[i, j] = kl_divergence(kde_models[i], kde_models[j], x_min, x_max)

print("KL散度矩阵（行是P，列是Q）：")
print(kl_matrix)

关键说明： #

• 积分区间取所有数据的极值并扩展微小边界，确保覆盖所有可能的取值范围
• 添加1e-10的极小值避免出现除以0或对数无意义的情况
• KL散度不满足对称性，即$D_{KL}(P||Q) \neq D_{KL}(Q||P)$，需注意顺序

三、验证分布合理性（可选） #

可以通过绘制密度曲线验证KDE模型是否符合预期：

import matplotlib.pyplot as plt

x = np.linspace(x_min, x_max, 1000)
for idx, kde in enumerate(kde_models):
    plt.plot(x, kde.evaluate(x), label=f"特征{idx}")
plt.legend()
plt.xlabel("Jaccard Index")
plt.ylabel("概率密度")
plt.title("各特征Jaccard指数的核密度分布")
plt.show()

内容的提问来源于stack exchange，提问作者Dwight Dinkins