我最近在搭建一个以12个特征输入、预测输出功率的GRU模型时，遇到了一个特别诡异的问题：无论怎么调整模型结构（层数1-5层、神经元50-512个）、训练参数（batch size、epoch数、优化器、激活函数），甚至添加Dropout、L2正则化、增减Dense层，模型的训练结果都完全一致——loss和val_loss在前2个epoch骤降后就趋于平稳，只有val_loss有小幅波动，这完全不符合预期。

后来我才揪出了问题的核心：测试数据集的规模远小于训练集，这正是之前模型效果极差且参数调整无效的主要原因。

针对这个问题，我对模型做了针对性优化，调整后的方案如下：

• 加入L2正则化（权重=0.0001）
• 新增两个无激活函数的Dense层，分别包含3个和5个节点
• 在第2和第3层GRU中加入Dropout（概率=0.1）
• 将batch size降至1000
• 损失函数替换为mae

完整优化后代码 #

import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from google.colab import files
from tensorboardcolab import TensorBoardColab, TensorBoardColabCallback
tbc=TensorBoardColab() # Tensorboard
from keras.layers.core import Dense
from keras.layers.recurrent import GRU
from keras.models import Sequential
from keras.callbacks import EarlyStopping
from keras import regularizers
from keras.layers import Dropout

df10=pd.read_csv('/content/drive/My Drive/Isolation Forest/IF 10 PERCENT.csv',index_col=None)
df2_10= pd.read_csv('/content/drive/My Drive/2019 Dataframe/2019 10minutes IF 10 PERCENT.csv',index_col=None)

X10_train= df10[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]
X10_train=X10_train.values
y10_train= df10['Power_kW']
y10_train=y10_train.values

X10_test= df2_10[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]
X10_test=X10_test.values
y10_test= df2_10['Power_kW']
y10_test=y10_test.values

# scaling values for model
x_scale = MinMaxScaler()
y_scale = MinMaxScaler()
X10_train= x_scale.fit_transform(X10_train)
y10_train= y_scale.fit_transform(y10_train.reshape(-1,1))
X10_test= x_scale.fit_transform(X10_test)
y10_test= y_scale.fit_transform(y10_test.reshape(-1,1))

X10_train = X10_train.reshape((-1,1,12))
X10_test = X10_test.reshape((-1,1,12))

Early_Stop=EarlyStopping(monitor='val_loss', patience=3 , mode='min',restore_best_weights=True)

# creating model using Keras
model10 = Sequential()
model10.add(GRU(units=200, return_sequences=True, input_shape=(1,12),activity_regularizer=regularizers.l2(0.0001)))
model10.add(GRU(units=100, return_sequences=True))
model10.add(Dropout(0.1))
model10.add(GRU(units=50))
model10.add(Dropout(0.1))
# 新增无激活函数的Dense层
model10.add(Dense(units=5))
model10.add(Dense(units=3))
model10.add(Dense(units=1, activation='linear'))
model10.compile(loss=['mae'], optimizer='adam',metrics=['mae'])
model10.summary()

history10=model10.fit(X10_train, y10_train, batch_size=1000,epochs=100,validation_split=0.1, verbose=1, callbacks=[TensorBoardColabCallback(tbc),Early_Stop])

score = model10.evaluate(X10_test, y10_test)
print('Score: {}'.format(score))

y10_predicted = model10.predict(X10_test)
y10_predicted = y_scale.inverse_transform(y10_predicted)
y10_test = y_scale.inverse_transform(y10_test)

plt.scatter( df2_10['WindSpeed_mps'], y10_test, label='Measurements',s=1)
plt.scatter( df2_10['WindSpeed_mps'], y10_predicted, label='Predicted',s=1)
plt.legend()
plt.savefig('/content/drive/My Drive/Figures/we move on curve6 IF10.png')
plt.show()

内容的提问来源于stack exchange，提问作者AliY