基于TensorFlow的迁移学习MIMO深度宽神经网络：足球预测准确率突破60%的优化建议

问题背景 #

我在做足球比赛结果预测时，模型准确率卡在60%左右——训练到30轮后准确率就不再提升，增加网络层数已经把高方差问题解决了，但还是没法突破准确率瓶颈。训练曲线表现为：训练准确率到30轮后平稳维持在60%，验证损失后期也不再明显下降。

以下是当前实现代码：

import pandas as pd
import numpy as np
import tensorflow as tf
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_absolute_error
import matplotlib.pyplot as plt

file_name = 'E2.csv'
# Define a function to convert time to text
def convert_time_to_text(value):
    return str(value)

# Specify the converters dictionary
converters = {'Time': convert_time_to_text}

# Read the CSV file with converters
data = pd.read_csv(file_name, converters=converters)

# Display the first few rows of your dataset to check the column names
print(data.head())
print(data.columns)
print(data.dtypes)
print(data.describe())
print(data.info())

# Define the feature columns and target columns
text_categorical_features = ["Time", "HoTe", "AwTe", "Ref"]
integer_target_columns = ["FTHG", "FTAG", "HTHG", "HTAG", "HS", "AS", "HST", "AST", "HF", "AF", "HC", "AC", "HY", "AY", "HR", "AR", "HBP", "ABP", "HTTG", "FTTG", "TS", "TST", "TC", "TF", "TY", "TR"]
y_columns = ['FTHG', 'FTAG', 'FTR', 'HTHG', 'HTAG', 'HTR', 'HS', 'AS', 'HST', 'AST', 'HF', 'AF', 'HC', 'AC', 'HY', 'AY', 'HR', 'AR', 'HBP', 'ABP', 'HTTG', 'FTTG', 'TS', 'TST', 'TC', 'TF', 'TY', 'TR']
text_categorical_target_columns = ["FTR", "HTR"]

# One-hot encode the 'Time' column
X_encoded_time = pd.get_dummies(data[['Time']])
X_encoded_time.columns = ['Time_' + str(col) for col in X_encoded_time.columns]  # Add a prefix to the column names
X_encoded_time.reset_index(drop=True, inplace=True)  # Reset the index for proper concatenation

# Combine one-hot-encoded 'Time' with other text-categorical features
X_encoded = pd.concat([data.drop(['Time'], axis=1), X_encoded_time, pd.get_dummies(data[text_categorical_features])], axis=1)
X_encoded = X_encoded.drop(integer_target_columns, axis=1)
X_encoded = X_encoded.drop(["HoTe", "AwTe", "Ref", "FTR", "HTR"], axis=1)
# Replace 'True' with 1 and 'False' with 0 in X_encoded
X_encoded = X_encoded.map(lambda x: 1 if x == True else (0 if x == False else x))

# One-hot encode the text-categorical target columns
y_encoded = pd.get_dummies(data[text_categorical_target_columns])
y_encoded_copy = y_encoded
y_encoded_copy = y_encoded_copy.map(lambda x: 1 if x == True else (0 if x == False else x))
print("One-hot encoded dummies for y_encoded: ")
print(y_encoded)

# Define the scaler
scaler = MinMaxScaler(feature_range=(0, 1))

# Scale the input values of integer_target_columns
data[integer_target_columns] = scaler.fit_transform(data[integer_target_columns])
y_encoded = pd.concat([y_encoded, data[integer_target_columns]], axis=1)
# Replace 'True' with 1 and 'False' with 0 in y_encoded
y_encoded = y_encoded.map(lambda x: 1 if x == True else (0 if x == False else x))

# Combine one-hot-encoded targets with the original dataframe excluding text-categorical target columns
data_encoded = pd.concat([X_encoded, y_encoded], axis=1)

# Print the resulting dataframe
print("data_encoded: ")
print(data_encoded)

X = X_encoded
y = y_encoded
print("X: ")
print(X)
print("y: ")
print(y)

# Assuming X and y are your input and output dataframes
# Convert X and y to numpy arrays
X = X.values
y = y.values

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=42)

# Define the model
model = tf.keras.models.Sequential()

# Input layer
print("X.shape[1]: ")
print(X.shape[1])
model.add(tf.keras.layers.Input(shape=(X.shape[1],)))

# Hidden layers
model.add(tf.keras.layers.Dense(64, activation='relu'))
model.add(tf.keras.layers.Dense(32, activation='relu'))
model.add(tf.keras.layers.Dense(16, activation='relu'))

# Output layer
print("y.shape[1]: ")
print(y.shape[1])
model.add(tf.keras.layers.Dense(y.shape[1], activation='sigmoid'))

# Compile the model
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Train the model
# model.fit(X_train, y_train, epochs=50, batch_size=16, validation_split=0.1)
history = model.fit(X_train, y_train, epochs=50, batch_size=16, validation_split=0.1)

# Save the model
model.save('my_model.h5')

# Access training history
training_accuracy = history.history['accuracy']
validation_loss = history.history['val_loss']

# Save and reload the model for making predictions
model.save('my_model.h5')
model = tf.keras.models.load_model('my_model.h5')

# Define the new data
new_data = pd.DataFrame({'Time': ['15:00'], 'HoTe': ['Blackpool'], 'AwTe': ['Burton'], 'Ref': ['S Stockbridge']})

# One-hot encode the 'Time' column
new_data_encoded_time = pd.get_dummies(new_data[['Time']])
new_data_encoded_time.columns = ['Time_' + str(col) for col in new_data_encoded_time.columns]          # Add a prefix to the column names
new_data_encoded_time.reset_index(drop=True, inplace=True)  # Reset the index for proper concatenation

# Combine one-hot-encoded 'Time' with other text-categorical features
new_data_encoded = pd.concat([new_data.drop(['Time'], axis=1), new_data_encoded_time,         pd.get_dummies(new_data[text_categorical_features])], axis=1)
new_data_encoded = new_data_encoded.reindex(columns=X_encoded.columns, fill_value=0)  # Reorder     columns to match the order in X_train_df.columns
new_data_encoded = new_data_encoded.astype(np.float32)  # Convert NumPy array to float32
print(" new_data_encoded: ")
print(new_data_encoded)

# Make predictions on the new_data_encoded
prediction = model.predict(new_data_encoded)
print("prediction: ")
print(prediction)
print(prediction.shape)

# Inverse transform the one-hot encoded part of the prediction output
def inverse_transform_prediction(prediction, text_categorical_target_columns, y_encoded_copy, integer_target_columns, scaler):
    # Extract the one-hot encoded part of the prediction for target columns
    num_columns = len(y_encoded_copy.columns)
    y_pred_one_hot = prediction[:, :num_columns]

    # Inverse transform the scaled integer predictions
    y_pred_integer_scaled = prediction[:, num_columns:]
    y_pred_integer = scaler.inverse_transform(y_pred_integer_scaled)

    # Map integer values to text values for target columns using y_encoded_copy
    y_pred_text_target_categorical = [
        [
            y_encoded_copy.columns[i] for i in range(len(y_encoded_copy.columns) // 2) if row[i] == 1
        ] +
        [
            y_encoded_copy.columns[i] for i in range(len(y_encoded_copy.columns) // 2, len(y_encoded_copy.columns)) if
            row[i] == 1
        ]
        for row in np.round(y_pred_one_hot)
    ]

    # Extract the last character from 'FTR' and 'HTR' columns
    ftr_htr_cols = ['FTR', 'HTR']
    y_pred_text_target_categorical = [val[-1] if col in ftr_htr_cols else val for val, col in
                                   zip(y_pred_text_target_categorical, y_encoded_copy)]

    # Convert the predicted integer values to text values for integer_target_columns
    y_pred_text_target_integer = [
        [str(round(value)) for value in row]
        for row in y_pred_integer
    ]

    # Concatenate the categorical and integer predictions
    y_pred_text_target = [
        cat + intg for cat, intg in zip(y_pred_text_target_categorical, y_pred_text_target_integer)
    ]

    return y_pred_text_target

# Use the function to inverse transform the predictions
y_pred_text = inverse_transform_prediction(prediction, text_categorical_target_columns, y_encoded_copy, integer_target_columns, scaler)

# Display the inverse transformed predictions
print("Inputs for Prediction:")
print(new_data)
print("Inverse Transformed Text Predictions:")
print(y_pred_text)

数据集样例：

Time HoTe AwTe FTHG FTAG FTR ... TS TST TC TF TY TR
0 15:00 Barnsley Port Vale 7 0 H ... 25 12 9 29 8 0
1 15:00 Blackpool Burton 2 0 H ... 24 7 11 22 5 0
2 15:00 Bolton Lincoln 3 0 H ... 22 4 11 35 6 0
3 15:00 Cambridge Oxford 2 0 H ... 20 9 12 19 5 0
4 15:00 Carlisle Fleetwood Town 1 1 D ... 28 11 7 26 5 0

[5 rows x 32 columns]

特征工程优化技巧 #

1. 拆分多输出任务，匹配合适的损失函数 #

当前模型把胜负分类、比分回归等所有任务混在一起用sigmoid输出和二元交叉熵损失，这会混淆不同任务的学习目标：

• 将**胜负结果（FTR、HTR）**拆分为独立的多分类任务，用softmax输出层，损失函数用categorical_crossentropy（独热编码时）或sparse_categorical_crossentropy（标签编码时）
• 将进球数、射门数等计数型目标拆分为回归任务，用线性输出层，损失函数用MSE或MAE
• 改用Keras的多输出模型架构，让每个任务的分支学习对应模式，提升整体预测能力

2. 挖掘球队历史与交叉特征 #

现有数据仅包含单场基础信息，可新增以下特征：

• 球队近期战绩：主队/客队最近3/5/10场的胜率、场均进球/失球、场均射门数等滚动统计值
• 交锋历史：两队过去N次交手的胜率、进球差、平局率
• 交叉特征：主队场均进球 × 客队场均失球、主队主场胜率 - 客队客场胜率这类融合双方特点的特征
• 时间特征：将Time转换为午场/晚场标识，若有日期信息可区分周中/周末赛事

3. 优化类别特征编码方式 #

当前对球队、裁判用独热编码，易导致特征维度爆炸且无法捕捉类别关联：

• 采用目标编码：对每个球队，用其历史比赛的平均结果（如主队胜率、场均进球）作为编码值，既压缩维度又融入任务信息
• 采用嵌入层：将球队、裁判等类别特征映射为低维向量，让神经网络自主学习类别间的相似性（需将类别特征转为整数编码后输入嵌入层）

4. 特征筛选与降维 #

• 计算特征与各目标变量的相关性，剔除相关性极低的冗余特征（如部分裁判的独热编码）
• 用PCA、互信息法或树模型的特征重要性筛选核心特征，减少噪声干扰，提升模型效率

5. 优化数据预处理 #

• 对进球数、射门数这类长尾分布的计数特征，尝试对数变换降低极端值影响
• 若存在缺失值，用球队历史均值、联赛均值等合理方式填充，避免直接丢弃数据

6. 引入外部数据 #

若能获取，以下数据可大幅提升预测能力：

• 球队阵容信息：主力球员伤病、停赛情况，首发阵容平均身价
• 赛前赔率：博彩机构的赔率数据（包含市场对比赛结果的预期）
• 场地与天气：比赛场地类型、天气状况（如雨天、高温）

内容的提问来源于stack exchange，提问作者user15096845