用于多目标（非多标签）预测的更优模型/算法选型咨询

Hey there! Let's walk through how to tackle this multi-target regression task (important note: this is predicting two distinct continuous targets y1 and y2, not multi-label classification) using your sample data. First, let's make your dataset clear with code:

import pandas as pd

df = pd.DataFrame({
    'x1':[1,2,3],
    'x2':[2,3,4],
    'x3':[1,1,1],
    'y1':[1,2,1],
    'y2':[2,3,3]
})

Which gives us this structured dataset:

Now let's dive into the two approaches you mentioned, plus context on when to use each:

1. Train Separate XGBoost Models for Each Target #

This is the most straightforward approach—treat y1 and y2 as entirely independent tasks. You'll build two separate XGBoost regressors, each optimized for its own target variable.

Pros: #

• Super easy to implement and debug; you can tune hyperparameters specifically for each target (e.g., different learning rates or tree depths for y1 vs y2).
• Works well if y1 and y2 have little to no correlation with each other.

Example Code: #

from xgboost import XGBRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

# Split features and targets
X = df[['x1', 'x2', 'x3']]
y1 = df['y1']
y2 = df['y2']

# Split into train/test sets
X_train, X_test, y1_train, y1_test, y2_train, y2_test = train_test_split(X, y1, y2, test_size=0.2, random_state=42)

# Train model for y1
model_y1 = XGBRegressor(objective='reg:squarederror', random_state=42)
model_y1.fit(X_train, y1_train)
y1_pred = model_y1.predict(X_test)
print(f"MSE for y1: {mean_squared_error(y1_test, y1_pred):.4f}")

# Train model for y2
model_y2 = XGBRegressor(objective='reg:squarederror', random_state=42)
model_y2.fit(X_train, y2_train)
y2_pred = model_y2.predict(X_test)
print(f"MSE for y2: {mean_squared_error(y2_test, y2_pred):.4f}")

2. Multi-Output Fully Connected Neural Network (MLP) #

This approach uses a single neural network that outputs both y1 and y2 simultaneously. The model shares lower-level feature extraction layers between the two targets, which can help if y1 and y2 are correlated (e.g., they depend on overlapping patterns in the input features).

Pros: #

• Captures potential relationships between y1 and y2, which might lead to better performance than separate models.
• Efficient in terms of computation if the targets share common features.

Example Code (using Keras): #

from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, Dense
from sklearn.preprocessing import StandardScaler

# Scale features (critical for neural network performance)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Build the multi-output model
input_layer = Input(shape=(3,))
hidden_layer = Dense(16, activation='relu')(input_layer)
hidden_layer = Dense(8, activation='relu')(hidden_layer)
output_layer = Dense(2)(hidden_layer)  # 2 outputs corresponding to y1 and y2

model = Model(inputs=input_layer, outputs=output_layer)
model.compile(optimizer='adam', loss='mse')

# Train the model
model.fit(X_scaled, df[['y1', 'y2']], epochs=50, batch_size=1, verbose=1)

# Generate predictions
predictions = model.predict(X_scaled)
y1_pred_nn = predictions[:, 0]
y2_pred_nn = predictions[:, 1]

Quick Decision Tip: #

• If your targets are uncorrelated or you want maximum control over each prediction task, stick with separate XGBoost models.
• If your targets are correlated and you have enough data to train a neural network effectively, the multi-output MLP might deliver better overall results.

内容的提问来源于stack exchange，提问作者Garvey