为何调用已保存的WhiteWine_Quality_Predictor模型后准确率大幅提升？

实验代码与结果 #

初始模型训练与测试 #

import pandas as pd
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, f1_score

whitewine_data = pd.read_csv('winequality-white.csv', 
delimiter=';')

variables = ['alcohol_cat', 'alcohol', 'sulphates', 'density', 
'total sulfur dioxide', 'citric acid', 'volatile acidity', 
'chlorides']

X = whitewine_data[variables]
y = whitewine_data['quality']
X_train, X_test, y_train, y_test = train_test_split(X, y, 
test_size=0.2)

model = DecisionTreeClassifier()
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
f1 = f1_score(y_test, y_pred, average='weighted')

predictions = model.predict([[0.27, 0.36, 0.045, 170, 1.001, 
0.45, 8.9, 0]])
print(f'Predicted Output: {predictions}')
print(f'Accuracy: {accuracy * 100}%')
print(f'F1 Score: {f1 * 100}% ')

该初始模型的准确率为57%

模型创建与保存 #

whitewine_data = pd.read_csv('winequality-white.csv', 
delimiter=';')

# Variables to be dropped from the data set - NOT THE INPUT 
VARIABLES
variables = ['fixed acidity', 'residual sugar', 'free sulfur 
dioxide', 'pH', 'quality', 'isSweet']

X = whitewine_data.drop(variables, axis=1)
y = whitewine_data['quality']

X_train, X_test, y_train, y_test = train_test_split(X, y, 
test_size=0.2)

model = DecisionTreeClassifier()
model.fit(X_train, y_train)

joblib.dump(model, 'WhiteWine_Quality_Predictor.joblib')

创建并保存模型

加载保存的模型并测试 #

whitewine_data = pd.read_csv('winequality-white.csv', 
delimiter=';') 

variables = ['volatile acidity', 'citric acid', 'chlorides', 
'total sulfur dioxide', 'density', 'sulphates', 'alcohol', 
'alcohol_cat']

X_test = whitewine_data[variables]
y_test = whitewine_data['quality']  

model = joblib.load('WhiteWine_Quality_Predictor.joblib')

y_pred = model.predict(X_test)

f1 = f1_score(y_test, y_pred, average='weighted')
accuracy = accuracy_score(y_test, y_pred)
predictions = model.predict([[0.27, 0.36, 0.045, 170, 1.001, 
0.45, 10.9, 3]])

print(f'F1 Score: {f1 * 100}%')
print(f'Model Accuracy: {accuracy * 100}%')
print(f'Predicted Output: {predictions}')

调用已保存的模型后，准确率达到92%

准确率飙升的原因分析 #

这一现象的核心是测试逻辑错误导致数据泄露，具体拆解：

1. 初始模型的测试是真实泛化能力：第一个代码中通过train_test_split将数据集按8:2拆分，测试集是完全未参与训练的20%陌生数据，57%的准确率是模型对未知样本的真实预测能力。

2. 加载模型后的测试存在数据泄露：第三个代码没有使用拆分后的独立测试集，而是直接将整个数据集作为测试输入。而保存的模型是用80%的训练数据训练的，这意味着测试数据里有80%是模型已经见过的训练样本。决策树本身容易过拟合，对见过的样本预测准确率极高，最终把整体准确率拉高到92%，但这不是模型的真实泛化能力。

3. 验证方法：如果在加载模型后，使用和初始模型相同的拆分逻辑（比如固定random_state保证拆分一致），用20%的陌生测试集评估，得到的准确率会和初始模型的57%接近，不会出现大幅飙升。

内容的提问来源于stack exchange，提问作者Henry