如何用Python绘制自定义阈值范围的MLP音频分类ROC曲线

问题背景 #

我正在使用MLP进行音频分类，现有代码可正常绘制ROC曲线并获取各类别最优阈值，但无法针对用户自定义的阈值范围绘制ROC曲线，求实现方法。

原运行正常的代码如下：

# Compute ROC curve and ROC area for each class
fpr = dict()
tpr = dict()
roc_auc = dict()
optimal_idx = dict()
optimal_threshold = dict()
for i in range(num_labels):
        fpr[i], tpr[i], thres = roc_curve(Y_test[:, i], Y_pred[:, i])
        roc_auc[i] = auc(fpr[i], tpr[i])
        # print(thres)
        optimal_idx[i] = np.argmax(tpr[i] - fpr[i])
        optimal_threshold[i] = thres[optimal_idx[i]]
        print(f'Threshold value for class{i}:', optimal_threshold[i])
      
        # Compute micro-average ROC curve and ROC area
fpr["micro"], tpr["micro"], thres = roc_curve(Y_test.ravel(), Y_pred.ravel())
roc_auc["micro"] = auc(fpr["micro"], tpr["micro"])
  
    # Compute macro-average ROC curve and ROC area
    # First aggregate all false positive rates
all_fpr = np.unique(np.concatenate([fpr[i] for i in range(num_labels)]))

      # Then interpolate all ROC curves at this points
mean_tpr = np.zeros_like(all_fpr)
  
for i in range(num_labels):
      mean_tpr += np.interp(all_fpr, fpr[i], tpr[i])
    
    # Finally average it and compute AUC
mean_tpr /= num_labels

fpr["macro"] = all_fpr
tpr["macro"] = mean_tpr
roc_auc["macro"] = auc(fpr["macro"], tpr["macro"])
      # Plot all ROC curves
lw=2
plt.figure()
plt.plot(
      fpr["micro"],
      tpr["micro"],
      label="micro-average ROC curve (area = {0:0.2f})".format(roc_auc["micro"]),
      color="deeppink",
      linestyle=":",
      linewidth=4,)

plt.plot(
      fpr["macro"],
      tpr["macro"],
      label="macro-average ROC curve (area = {0:0.2f})".format(roc_auc["macro"]),
      color="navy",
      linestyle=":",
      linewidth=4,)
# from itertools import cycle
colors = cycle(["aqua", "darkorange", "cornflowerblue"])
for i, color in zip(range(num_labels), colors):
      plt.plot(
          fpr[i],
          tpr[i],
          color=color,
          lw=lw,
          label="ROC curve of class {0} (area = {1:0.2f})".format(i, roc_auc[i]),
      )

plt.plot([0, 1], [0, 1], "k--", lw=lw)
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")
plt.title("ROC for MLP")
plt.legend(loc="lower right")
plt.show()

实现方法 #

核心逻辑是手动遍历自定义阈值，逐个计算对应的FPR和TPR，再用这些结果绘制ROC曲线，具体步骤如下：

1. 定义自定义阈值范围 #

可以用numpy生成指定区间的阈值数组，或者直接传入自定义的阈值列表：

import numpy as np

# 示例1：生成0.1到0.9，步长0.1的阈值
custom_thresholds = np.linspace(0.1, 0.9, 9)
# 示例2：直接传入自定义阈值列表
# custom_thresholds = [0.2, 0.4, 0.5, 0.7, 0.8]

2. 计算自定义阈值对应的FPR和TPR #

遍历每个类别，针对自定义阈值逐一计算混淆矩阵，推导FPR和TPR：

from sklearn.metrics import confusion_matrix

fpr_custom = dict()
tpr_custom = dict()

for i in range(num_labels):
    y_true = Y_test[:, i]  # 当前类别的真实标签
    y_score = Y_pred[:, i]  # 当前类别的预测得分
    
    fpr_list = []
    tpr_list = []
    
    for thresh in custom_thresholds:
        # 根据阈值生成二分类预测标签
        y_pred = (y_score >= thresh).astype(int)
        # 计算混淆矩阵（TN, FP, FN, TP）
        tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()
        # 计算FPR和TPR（避免分母为0的情况）
        fpr = fp / (fp + tn) if (fp + tn) != 0 else 0.0
        tpr = tp / (tp + fn) if (tp + fn) != 0 else 0.0
        
        fpr_list.append(fpr)
        tpr_list.append(tpr)
    
    # 转换为numpy数组便于后续绘图和计算
    fpr_custom[i] = np.array(fpr_list)
    tpr_custom[i] = np.array(tpr_list)

3. 计算自定义阈值曲线的AUC（可选） #

如果需要评估自定义阈值曲线的AUC，直接使用auc函数：

from sklearn.metrics import auc

roc_auc_custom = dict()
for i in range(num_labels):
    roc_auc_custom[i] = auc(fpr_custom[i], tpr_custom[i])

4. 绘制自定义阈值的ROC曲线 #

替换原绘图逻辑中的fpr和tpr为自定义计算的结果，还可以添加标记点突出阈值位置：

import matplotlib.pyplot as plt
from itertools import cycle

lw = 2
plt.figure(figsize=(8, 6))

colors = cycle(["aqua", "darkorange", "cornflowerblue"])
for i, color in zip(range(num_labels), colors):
    plt.plot(
        fpr_custom[i],
        tpr_custom[i],
        color=color,
        lw=lw,
        marker='o',  # 用圆点标记每个自定义阈值对应的点
        label=f"Custom ROC of class {i} (AUC = {roc_auc_custom[i]:0.2f})",
    )

# 绘制随机猜测的基准线
plt.plot([0, 1], [0, 1], "k--", lw=lw)
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")
plt.title("ROC for MLP (Custom Thresholds)")
plt.legend(loc="lower right")
plt.show()

5. 微平均/宏平均的自定义阈值ROC（可选） #

如果需要全局的微平均或宏平均曲线，逻辑如下：

• 微平均：将所有样本的真实标签和预测得分拉平，再遍历自定义阈值计算全局FPR和TPR
• 宏平均：对每个类别的自定义FPR和TPR取算术平均

示例微平均实现：

# 微平均自定义阈值ROC
y_true_flat = Y_test.ravel()
y_score_flat = Y_pred.ravel()

fpr_micro_custom = []
tpr_micro_custom = []

for thresh in custom_thresholds:
    y_pred_flat = (y_score_flat >= thresh).astype(int)
    tn, fp, fn, tp = confusion_matrix(y_true_flat, y_pred_flat).ravel()
    fpr = fp / (fp + tn) if (fp + tn) != 0 else 0.0
    tpr = tp / (tp + fn) if (tp + fn) != 0 else 0.0
    fpr_micro_custom.append(fpr)
    tpr_micro_custom.append(tpr)

fpr_micro_custom = np.array(fpr_micro_custom)
tpr_micro_custom = np.array(tpr_micro_custom)
roc_auc_micro_custom = auc(fpr_micro_custom, tpr_micro_custom)

# 绘制微平均曲线
plt.plot(
    fpr_micro_custom,
    tpr_micro_custom,
    color="deeppink",
    linestyle=":",
    linewidth=4,
    marker='s',
    label=f"Micro-average Custom ROC (AUC = {roc_auc_micro_custom:0.2f})"
)

内容的提问来源于stack exchange，提问作者A. Gehani