逻辑回归执行异常：奇异矩阵、Hessian问题及p值NaN求助

Let’s walk through your problems one by one—these are super common hurdles with statsmodels’ Logit implementation, so you’re not stuck alone here.

First: Fix the ValueError: Pandas data cast to numpy dtype of object #

Even though you added .astype(float) to X1, there’s still something non-numeric sneaking into your features. Here’s what to check:

• Inspect data types: Run X1.info() to see if any columns are still listed as object type. These could be hidden string values, mixed data types, or unhandled categorical variables (like "Yes"/"No" instead of 1/0).
• Check for missing values: Run X1.isna().sum()—missing values can sometimes force columns into object dtype even after .astype(float). Decide whether to drop rows with missing values or impute them (mean/median for numeric, mode for categorical).
• Encode categorical variables: Statsmodels doesn’t automatically handle string-based categories. Use one-hot encoding for unordered categories:

  X1 = pd.get_dummies(X1, drop_first=True)  # drop_first avoids dummy variable trap
  

  For ordered categories, use label encoding instead.

Next: Resolve the Singular Matrix/Hessian Errors #

You mentioned your dataset has no correlation, but singular matrices usually stem from a few hidden issues:

• Missing intercept term: Statsmodels’ Logit does NOT include an intercept by default. Skipping this can lead to numerical instability. Add it with:

  X1 = sm.add_constant(X1)
  

• Complete separation: This happens when a feature’s values perfectly predict your target (y1). For example, every time FeatureA = 1, y1 = 1, and when FeatureA = 0, y1 = 0. Check for this with cross-tabs:

  for col in X1.columns:
      print(pd.crosstab(X1[col], y1))
  

  If you find such a feature, remove it—your model can’t learn anything meaningful from it, and it breaks the Hessian matrix.
• Hidden multicollinearity: Even if you don’t see obvious correlations, you might have features that are linear combinations of each other (e.g., FeatureC = FeatureA + FeatureB). Use Variance Inflation Factor (VIF) to detect this:

  from statsmodels.stats.outliers_influence import variance_inflation_factor
  vif_data = pd.DataFrame()
  vif_data["feature"] = X1.columns
  vif_data["VIF"] = [variance_inflation_factor(X1.values, i) for i in range(X1.shape[1])]
  print(vif_data)
  

  A VIF score > 10 indicates severe multicollinearity—drop the highest VIF feature and recheck.
• Too many features vs. sample size: If you have more features than rows, or close to it, the matrix becomes singular. Reduce features via feature selection (e.g., recursive elimination) or dimensionality reduction (e.g., PCA).

Finally: Fix "non" p-values #

The "non" p-values happen because optimization methods like Powell or BFGS can’t converge to a stable solution when the Hessian is singular. Here’s how to fix it:

• Use regularization: The fit_regularized method adds penalties to your model, which stabilizes the Hessian and allows convergence. Try L1 or L2 regularization:

  model = sm.Logit(y1, X1.astype(float))
  result = model.fit_regularized(method='l1', alpha=0.3, maxiter=1000)
  print(result.params)
  

  Adjust alpha (higher = stronger regularization) until you get stable results.
• Switch optimization methods: Try more robust optimizers like Newton-Raphson or conjugate gradient:

  result = model.fit(method='newton', maxiter=100)  # Newton method is good for well-behaved data
  # Or try conjugate gradient
  result = model.fit(method='cg', maxiter=200)
  

  These methods handle numerical instability better than Powell/BFGS in some cases.

Step-by-Step Action Plan #

1. Clean your data: Fix data types, handle missing values, encode categoricals.
2. Add the intercept term to X1.
3. Check for complete separation and multicollinearity—remove problematic features.
4. First try fitting with method='newton'. If that fails, use fit_regularized with L1/L2 regularization.

内容的提问来源于stack exchange，提问作者Sim_Demo