如何在Pandas DataFrame中执行多重共线性检验并剔除VIF>5的变量？

Hey there! Let's work through this together—since you're new to these tools, we'll break down what's wrong with the original code, how to use it correctly, and why each step matters.

First: What's the Issue with the Original Code? #

The main problems are:

• You can't modify a range object (which is what variables = range(X.shape[1]) creates) with del—that's why you get an error when trying to drop columns. We need to use a mutable list instead.
• You should not pass your entire dataset (including the target variable) to X. VIF measures collinearity between feature variables (predictors), so X should only include columns you plan to use in your model (exclude the target column, like y).

Corrected Code #

Here's the fixed version of the function, with comments explaining each part:

from statsmodels.stats.outliers_influence import variance_inflation_factor
import pandas as pd

def calculate_vif(X, thresh=5.0):
    # Convert column indices to a mutable list (instead of range)
    variables = list(range(X.shape[1]))
    dropped = True
    
    while dropped:
        dropped = False
        # Calculate VIF for each remaining variable
        vif_scores = [
            variance_inflation_factor(X.iloc[:, variables].values, ix) 
            for ix in range(len(variables))
        ]
        # Find the variable with the highest VIF
        max_vif = max(vif_scores)
        max_loc = vif_scores.index(max_vif)
        
        if max_vif > thresh:
            print(f"Dropping '{X.columns[variables[max_loc]]}' (VIF: {max_vif:.2f})")
            # Remove the index of the variable from our list
            del variables[max_loc]
            dropped = True
    
    print("\nRemaining variables (VIF ≤ 5):")
    remaining_cols = X.columns[variables].tolist()
    print(remaining_cols)
    return X[remaining_cols]

How to Use This Correctly #

Let's walk through a real example with a sample dataset:

1. Prepare your feature matrix:
   Suppose your dataset is stored in a pandas DataFrame called df, and your target variable is named 'target'. First, separate your features from the target:

   # Drop the target column to get only features
   X = df.drop('target', axis=1)
   

   Important: Ensure all columns in X are numeric. If you have categorical variables (like strings), you'll need to encode them first (e.g., using one-hot encoding with pd.get_dummies()).

2. Run the VIF function:

   # Get features with VIF ≤5
   X_cleaned = calculate_vif(X)
   

   This will print each column it drops (with its VIF score) and return a new DataFrame with only the columns that pass the VIF threshold.

Key Notes for Beginners #

• What is VIF?: Variance Inflation Factor measures how much the variance of a coefficient is inflated due to collinearity with other features. A VIF >5 means significant collinearity that can hurt model performance.
• Why not pass the whole dataset?: Including the target variable would calculate VIF between features and the target, which isn't what we want—we only care about collinearity between predictors.
• Data types matter: Statsmodels can't calculate VIF on non-numeric data, so always preprocess categorical columns before using this function.

内容的提问来源于stack exchange，提问作者Aakash Basu