使用bestglm进行逻辑回归子集选择的疑问与警告解析

构造的数据集 #

a <- rnorm(10,5,1)
b <- rnorm(10,5,1)
c <- rnorm(10,5,1)
d <- rnorm(10,5,1)
e <- rnorm(10,5,1)
f <- rnorm(10,5,1)
g <- rnorm(10,5,1)
h <- sample(c(0,1), replace=TRUE, size=10) # Random 1s and 0s

df = data.frame(a,b,c,d,e,f,g,h)

执行的代码 #

library(bestglm)

bestglm(df,family=binomial,IC="AIC",method="exhaustive")

输出结果 #

AIC
BICq equivalent for q in (0.370228512040497, 0.759746926627138)
Best Model:
              Estimate Std. Error       z value  Pr(>|z|)
(Intercept) 1015.83048  1975540.4  0.0005142038 0.9995897
c            158.60524   243304.6  0.0006518794 0.9994799
d             77.54088   125534.0  0.0006176885 0.9995072
e           -279.63330   456645.4 -0.0006123642 0.9995114
f           -152.81443   246860.1 -0.0006190326 0.9995061

There were 34 warnings (use warnings() to see them)
> warnings()
Warning messages:
1: glm.fit: fitted probabilities numerically 0 or 1 occurred
2: glm.fit: algorithm did not converge
3: glm.fit: fitted probabilities numerically 0 or 1 occurred
4: glm.fit: fitted probabilities numerically 0 or 1 occurred
5: glm.fit: algorithm did not converge
6: glm.fit: fitted probabilities numerically 0 or 1 occurred
7: glm.fit: algorithm did not converge
8: glm.fit: fitted probabilities numerically 0 or 1 occurred
9: glm.fit: algorithm did not converge
10: glm.fit: fitted probabilities numerically 0 or 1 occurred
11: glm.fit: fitted probabilities numerically 0 or 1 occurred
12: glm.fit: fitted probabilities numerically 0 or 1 occurred
13: glm.fit: algorithm did not converge
14: glm.fit: fitted probabilities numerically 0 or 1 occurred
15: glm.fit: algorithm did not converge
16: glm.fit: fitted probabilities numerically 0 or 1 occurred
17: glm.fit: algorithm did not converge
18: glm.fit: fitted probabilities numerically 0 or 1 occurred
19: glm.fit: algorithm did not converge
20: glm.fit: fitted probabilities numerically 0 or 1 occurred
21: glm.fit: fitted probabilities numerically 0 or 1 occurred
22: glm.fit: fitted probabilities numerically 0 or 1 occurred
23: glm.fit: algorithm did not converge
24: glm.fit: fitted probabilities numerically 0 or 1 occurred
25: glm.fit: algorithm did not converge
26: glm.fit: fitted probabilities numerically 0 or 1 occurred
27: glm.fit: algorithm did not converge
28: glm.fit: fitted probabilities numerically 0 or 1 occurred
29: glm.fit: algorithm did not converge
30: glm.fit: fitted probabilities numerically 0 or 1 occurred
31: glm.fit: algorithm did not converge
32: glm.fit: fitted probabilities numerically 0 or 1 occurred
33: glm.fit: fitted probabilities numerically 0 or 1 occurred
34: glm.fit: fitted probabilities numerically 0 or 1 occurred

用户疑问：

• 无法理解这些警告的含义
• 不确定模型是否自动将h识别为目标变量

1. 目标变量的识别 #

bestglm默认将数据框的最后一列作为响应变量（目标变量）。你的数据框df最后一列是h，所以模型确实自动将h识别为目标变量，符合需求。如果需要指定其他列作为响应变量，可通过y参数明确指定，比如y=df$h，同时用X参数传入预测变量（X=df[,1:7]）。

2. 警告信息的含义与原因 #

（1）glm.fit: fitted probabilities numerically 0 or 1 occurred #

该警告表示模型计算的拟合概率数值上接近0或1，本质是出现了完全分离或准完全分离：

• 完全分离：存在某个预测变量的取值，能完美区分h=0和h=1的样本；
• 准完全分离：大部分样本能被某个预测变量完美区分，仅少数重叠。

你的数据集只有10个样本，预测变量却有7个，样本量远小于变量数，极易出现这种情况——模型为拟合数据，会将系数推至极大值，导致拟合概率趋近于0或1。

（2）glm.fit: algorithm did not converge #

逻辑回归采用迭代加权最小二乘法（IRLS）拟合模型，当系数趋近于无穷大时，迭代过程无法达到收敛阈值，就会抛出该警告。这通常是完全/准完全分离的直接后果。

3. 解决建议 #

• 增加样本量：样本量需远大于预测变量数（一般建议每个变量至少对应10-20个样本）；
• 减少预测变量：先通过相关性分析、单变量逻辑回归等方式筛选变量，去掉冗余变量后再做子集选择；
• 使用正则化方法：比如用glmnet包做L1正则化（Lasso）的逻辑回归，可处理分离问题，同时自动完成变量选择；
• 调整拟合参数：在bestglm中尝试设置maxit参数增加迭代次数，但这仅为临时方案，无法解决样本量不足或分离的根本问题。

内容的提问来源于stack exchange，提问作者Abubakar Popoola