基于神经网络的多元线性回归系数值相关技术咨询

Hey there! Let's unpack your question about multiple linear regression (MLR) coefficients in the context of your neural network regression setup. I’ll break this down clearly based on your specific model and data.

First, let’s ground ourselves in what MLR coefficients represent for your data. With 4 independent features (X₁ to X₄) and 1 dependent variable (y), the MLR model is:

ŷ = β₀ + β₁X₁ + β₂X₂ + β₃X₃ + β₄X₄

Here, β₀ is the intercept, and each βᵢ is the coefficient for feature Xᵢ. These coefficients tell you the marginal linear impact of each feature on the predicted value (interpretation shifts slightly if features are standardized vs. unstandardized).

Now, how does your trained neural network (1 hidden layer with 2 neurons, 1 output neuron) connect to these coefficients? It depends entirely on the activation function you used in the hidden layer.

Case 1: Your Neural Network Uses Nonlinear Activations (Most Common) #

If you used a nonlinear activation like ReLU, sigmoid, or tanh in the hidden layer (the standard choice for regression tasks where linear models fall short), your neural network is a nonlinear model. This means the relationship between your input features and the output ŷ isn’t a straight line—so there’s no direct 1:1 mapping between the NN’s weights/biases and MLR’s β coefficients.

The hidden layer’s nonlinear transformation warps the input features in a way that can’t be reduced to a simple linear combination. So you can’t extract "MLR-style coefficients" directly from your NN’s parameters here.

Case 2: Your Neural Network Uses Identity Activations (Rare) #

If you used an identity activation (no transformation, just pass the hidden layer output straight through) in the hidden layer, your NN is actually equivalent to a linear model—though it’s a convoluted way to build one. In this scenario, you can calculate equivalent MLR coefficients by combining the hidden layer and output layer weights:

Let’s define:

• Hidden layer neuron 1: h₁ = w₁₀ + w₁₁X₁ + w₁₂X₂ + w₁₃X₃ + w₁₄X₄
• Hidden layer neuron 2: h₂ = w₂₀ + w₂₁X₁ + w₂₂X₂ + w₂₃X₃ + w₂₄X₄
• Output layer: ŷ = v₀ + v₁h₁ + v₂h₂

Expanding the output gives:

ŷ = (v₀ + v₁w₁₀ + v₂w₂₀) + (v₁w₁₁ + v₂w₂₁)X₁ + (v₁w₁₂ + v₂w₂₂)X₂ + (v₁w₁₃ + v₂w₂₃)X₃ + (v₁w₁₄ + v₂w₂₄)X₄

Here, the intercept β₀ is v₀ + v₁w₁₀ + v₂w₂₀, and each βᵢ is v₁w₁ᵢ + v₂w₂ᵢ. That said, using an identity activation in a hidden layer defeats the purpose of using a neural network—you’d be better off just training an MLR model directly.

If You Want MLR Coefficients for Comparison or Analysis #

If your goal is to get actual MLR coefficients (to compare performance with your NN, or for interpretability), you can easily train an MLR model on your same dataset. Here’s a quick Python example using scikit-learn:

import pandas as pd
from sklearn.linear_model import LinearRegression

# Assume your data is stored in a DataFrame `df`, with first 4 columns as features, last as target
X = df.iloc[:, :4]  # Independent variables
y = df.iloc[:, 4]   # Dependent variable

# Train the MLR model
lr_model = LinearRegression()
lr_model.fit(X, y)

# Extract coefficients and intercept
print(f"MLR Intercept (β₀): {lr_model.intercept_}")
print(f"MLR Coefficients (β₁ to β₄): {lr_model.coef_}")

If you want to match the regularization you used in your NN, use a regularized linear model like Ridge (L2 regularization) or Lasso (L1 regularization) instead of LinearRegression.

If You Want "Feature Impact" Metrics From Your Neural Network #

If you’re looking for something similar to MLR coefficients (i.e., understanding how each feature affects predictions) but for your nonlinear NN, you can use these methods:

• Permutation Importance: Shuffle the values of one feature at a time, and measure how much your model’s prediction performance drops. A bigger drop means the feature is more impactful.
• SHAP Values: These quantify the exact contribution of each feature to individual predictions and overall model behavior—they’re great for interpretability in nonlinear models.
• Gradient-Based Importance: Calculate the average gradient of the output ŷ with respect to each input feature across your dataset. Higher absolute values mean the feature has a larger impact.

内容的提问来源于stack exchange，提问作者Neelabh Pant