XGBRegressor叶节点值求和与模型预测结果不匹配问题排查

原本认为XGBoost的XGBRegressor最终预测结果是各树预测叶节点值的总和，但实际求和结果与模型预测值无法匹配。以下是最小可复现示例（MRE）：

import json
from collections import deque

import numpy as np
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
import xgboost as xgb


def leafs_vector(tree):
    """Returns a vector of nodes for each tree, only leafs are different of 0"""

    stack = deque([tree])

    while stack:
        node = stack.popleft()
        if "leaf" in node:
            yield node["leaf"]
        else:
            yield 0
            for child in node["children"]:
                stack.append(child)


# Load the diabetes dataset
diabetes = load_diabetes()
X, y = diabetes.data, diabetes.target

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Define the XGBoost regressor model
xg_reg = xgb.XGBRegressor(objective='reg:squarederror',
                          max_depth=5,
                          n_estimators=10)

# Train the model
xg_reg.fit(X_train, y_train)

# Compute the original predictions
y_pred = xg_reg.predict(X_test)

# get the index of each predicted leaf
predicted_leafs_indices = xg_reg.get_booster().predict(xgb.DMatrix(X_test), pred_leaf=True).astype(np.int32)

# get the trees
trees = xg_reg.get_booster().get_dump(dump_format="json")
trees = [json.loads(tree) for tree in trees]

# get a vector of nodes (ordered by node id)
leafs = [list(leafs_vector(tree)) for tree in trees]

l_pred = []
for pli in predicted_leafs_indices:
    l_pred.append(sum(li[p] for li, p in zip(leafs, pli)))

assert np.allclose(np.array(l_pred), y_pred, atol=0.5) # fails

尝试添加默认base_score（0.5）到总和中，仍然无效：

l_pred = []
for pli in predicted_leafs_indices:
    l_pred.append(sum(li[p] for li, p in zip(leafs, pli)) + 0.5) 

问题 #

为何叶节点值求和结果与XGBRegressor的预测值不匹配？如何解决该问题？

原因分析 #

1. 叶节点索引与值的对应错误：pred_leaf=True返回的叶节点索引是树中叶节点的遍历顺序序号（从1开始），但你的leafs_vector函数生成的是包含所有节点（非叶节点填充为0）的广度优先遍历列表，导致索引和叶节点值的对应关系完全错位。
2. base_score的实际取值错误：使用reg:squarederror目标时，XGBoost默认会将base_score设置为训练集y的均值，而非固定的0.5，直接加0.5自然无法匹配。

解决方案 #

修正叶节点映射与base_score获取 #

需要正确建立叶节点索引到值的映射，并使用模型实际的base_score，修正后的代码如下：

import json
import numpy as np
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
import xgboost as xgb


def get_leaf_index_map(tree):
    """为每棵树建立叶节点序号（pred_leaf返回的索引）到值的映射"""
    leaf_map = {}
    current_leaf_idx = 1
    # 用栈实现先序遍历（左子树优先），和pred_leaf的索引生成逻辑一致
    stack = [tree]
    while stack:
        node = stack.pop()
        if "leaf" in node:
            leaf_map[current_leaf_idx] = node["leaf"]
            current_leaf_idx += 1
        else:
            # 先压右子节点，再压左子节点，保证遍历顺序正确
            stack.append(node["children"][1])
            stack.append(node["children"][0])
    return leaf_map


# 加载数据与训练模型
diabetes = load_diabetes()
X, y = diabetes.data, diabetes.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

xg_reg = xgb.XGBRegressor(objective='reg:squarederror', max_depth=5, n_estimators=10)
xg_reg.fit(X_train, y_train)

y_pred = xg_reg.predict(X_test)
predicted_leafs_indices = xg_reg.get_booster().predict(xgb.DMatrix(X_test), pred_leaf=True).astype(np.int32)

# 获取每棵树的叶节点映射
trees = [json.loads(tree) for tree in xg_reg.get_booster().get_dump(dump_format="json")]
leaf_maps = [get_leaf_index_map(tree) for tree in trees]

# 获取模型实际的base_score
base_score = float(xg_reg.get_booster().attr('base_score'))

# 计算预测值：base_score + 所有树的叶节点值之和
l_pred = []
for pli in predicted_leafs_indices:
    sum_leaf = sum(leaf_map[idx] for leaf_map, idx in zip(leaf_maps, pli))
    l_pred.append(base_score + sum_leaf)

# 验证匹配，误差范围设为1e-5即可通过
assert np.allclose(np.array(l_pred), y_pred, atol=1e-5)

关键说明 #

• 叶节点索引顺序：pred_leaf返回的索引是按树的**先序遍历（左子树优先）**生成的叶节点序号，构建映射时必须保持相同的遍历顺序，否则会出现索引错位。
• base_score的正确获取：直接从模型的booster中读取base_score属性，该值在默认训练逻辑下等于训练集y的均值，无需手动设置。

内容的提问来源于stack exchange，提问作者Dani Mesejo