如何用线性回归以外的方法量化各自变量对因变量的影响强度？

Great question! Linear regression's standardized betas are a solid go-to for comparing how much each independent variable moves the needle on your dependent variable—like in your sales example with TV, digital, and radio spend. But as you rightly pointed out, they only work well for linear relationships and break down when those assumptions don't hold.

Below are practical, model-agnostic and model-specific methods to quantify feature impact beyond linear regression:

• Tree-Based Models (Random Forest, XGBoost, LightGBM)
  These models come built-in with feature importance metrics, typically based on either Gini impurity reduction or mean decrease in accuracy. For example, in scikit-learn's RandomForestRegressor, you can access the feature_importances_ attribute right after training. That said, note that high-cardinality features can get overrated here.
  For more precise, interpretable impact (including non-linear relationships), use SHAP values or Partial Dependence Plots (PDPs). SHAP values are especially powerful—rooted in game theory, they calculate the exact contribution of each feature to individual predictions, which you can aggregate to get global importance scores. You can implement this with the shap library:

  import shap
  explainer = shap.TreeExplainer(your_trained_model)
  shap_values = explainer.shap_values(X_test)
  shap.summary_plot(shap_values, X_test)
  

• Permutation Importance
  This is a model-agnostic method that works with any model—linear or not. The idea is simple: shuffle the values of a single feature, then measure how much your model's performance (e.g., RMSE for regression, accuracy for classification) drops. The bigger the drop, the more important the feature is. It avoids the bias toward high-cardinality features that tree models can have.
  In scikit-learn, you can use sklearn.inspection.permutation_importance to compute this easily:

  from sklearn.inspection import permutation_importance
  result = permutation_importance(your_trained_model, X_test, y_test, n_repeats=10, random_state=42)
  

• Generalized Additive Models (GAMs)
  GAMs let you model non-linear relationships (each feature is fit with a smooth function) while keeping the interpretability of linear models. You can quantify feature impact by looking at:

  • The effective degrees of freedom (edf) of each smooth term (higher edf means more non-linear variation explained)
  • The proportion of variance explained by each feature
    The pygam library makes this straightforward—after fitting a model, you can plot each feature's smooth function and extract its statistical significance.

• LIME (Local Interpretable Model-agnostic Explanations)
  Another model-agnostic tool, LIME creates local linear approximations around individual predictions to explain which features drove that specific outcome. You can aggregate these local explanations to get a global sense of feature importance. It's great for explaining model behavior to non-technical stakeholders since it uses simple, interpretable terms.

• Mutual Information
  If you want to measure the strength of any relationship (linear or non-linear) between a feature and the target, mutual information is a great choice. It quantifies how much knowing one variable reduces uncertainty about the other. For regression tasks, use sklearn.feature_selection.mutual_info_regression; for classification, use mutual_info_classif. Unlike correlation metrics, it captures non-monotonic relationships too.

• Neural Network Interpretability Tools
  If you're using deep learning models, you can use:

  • Integrated Gradients: Calculates feature attribution by integrating gradients from a baseline input (e.g., all zeros) to the actual input, giving you a fair measure of each feature's contribution.
  • Layer-wise Relevance Propagation (LRP): Backpropagates the model's prediction score to the input layer, assigning a relevance score to each feature that indicates how much it influenced the output.

The best approach depends on your model type and what you need to measure—whether it's global feature importance, local impact on individual predictions, or capturing complex non-linear patterns. Combining multiple methods (like SHAP values + permutation importance) can give you a more robust, well-rounded view of feature impact.

内容的提问来源于stack exchange，提问作者Sharath G