能否修改DecisionTreeRegressor的MAE准则，以均值替代中位数计算？

Absolutely! You can absolutely adjust the DecisionTreeRegressor to use the mean (instead of the default median) for leaf node predictions when using the MAE criterion. Let me break down how to do this, along with some context on why the default uses median.

Background #

Scikit-learn's DecisionTreeRegressor uses the median for leaf nodes when criterion='mae' because mathematically, the median minimizes the mean absolute error (MAE). That said, if your use case specifically requires using the mean for leaf node predictions (even if it doesn't optimize MAE), you can easily override this behavior by subclassing the regressor.

Step-by-Step Implementation #

Here's how to create a modified version that uses the mean for leaf nodes:

from sklearn.tree import DecisionTreeRegressor
import numpy as np

class MeanMAEDecisionTreeRegressor(DecisionTreeRegressor):
    def __init__(self, **kwargs):
        # Force criterion to 'mae' since we're modifying its leaf behavior
        super().__init__(criterion='mae', **kwargs)
    
    def _compute_node_value(self, y):
        # Override the default median calculation with mean
        return np.mean(y)

Test the Modified Regressor #

Let's compare the original and modified versions with a sample dataset:

from sklearn.datasets import make_regression
from sklearn.metrics import mean_absolute_error

# Generate sample regression data
X, y = make_regression(n_samples=100, n_features=2, noise=0.1, random_state=42)

# Original MAE tree (uses median for leaf nodes)
original_regr = DecisionTreeRegressor(max_depth=2, criterion='mae')
original_regr.fit(X, y)
original_preds = original_regr.predict(X)
original_mae = mean_absolute_error(y, original_preds)
print(f"Original MAE (median leaves): {original_mae:.4f}")

# Modified tree (uses mean for leaf nodes)
modified_regr = MeanMAEDecisionTreeRegressor(max_depth=2)
modified_regr.fit(X, y)
modified_preds = modified_regr.predict(X)
modified_mae = mean_absolute_error(y, modified_preds)
print(f"Modified MAE (mean leaves): {modified_mae:.4f}")

Key Notes #

• What we changed: We only override the _compute_node_value method, which calculates the value stored in each leaf node. The original method returns the median of the node's target values; our version returns the mean.
• Split logic remains the same: The tree's split points are still determined using the original MAE-based logic (which relies on median splits). If you want to modify the split criteria to use mean-based MAE calculations, that would require deeper changes to scikit-learn's internal Cython tree code—this is more complex and generally not recommended unless you have a strong need for it.
• Tradeoff: Using the mean will result in a higher training MAE compared to the default median, since the median is the optimal value for minimizing MAE. Only use this modification if your business logic or specific requirements demand it.

内容的提问来源于stack exchange，提问作者Jacob