CNN-LSTM时序传感器动作识别验证准确率波动问题求解

本次任务使用时序传感器数据，数据集为类别不平衡数据集，共包含12类数据，目标是实现人体日常动作预测。

注意：LSTM输出直接接入最终输出层

con_l1 = tf.keras.layers.Conv2D(64, (5, 1), activation="relu")(input_layer) 
con_l2 = tf.keras.layers.Conv2D(64, (5, 1), activation="relu")(con_l1)
con_l3 = tf.keras.layers.Conv2D(64, (5, 1), activation="relu")(con_l2)
con_l4 = tf.keras.layers.Conv2D(64, (5, 1), activation="relu")(con_l3)
rl = Reshape((int(con_l4.shape[1]), int(con_l4.shape[2]) * int(con_l4.shape[3])))(con_l4)
lstm_l5 = tf.keras.layers.LSTM(128, activation='tanh',
                                recurrent_initializer=tf.keras.initializers.Orthogonal(seed=0), dropout=0.5,
                                       recurrent_dropout=0.25, return_sequences=True)(rl)
# required output of each cell to feed into second LSTM layer, so thats why return_sequences=True
        
lstm_l6 = tf.keras.layers.LSTM(128, activation='tanh',
                                           recurrent_initializer=tf.keras.initializers.Orthogonal(seed=1), dropout=0.5,
                                           recurrent_dropout=0.25, return_sequences=True)(lstm_l5)

学习率设置为每10轮衰减0.9，配套配置如下：

tf.keras.optimizers.Adam(learning_rate=0.001)

model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["acc"])
early_Stopping = tf.keras.callbacks.EarlyStopping(monitor='val_loss', mode = 'min', patience=10, restore_best_weights=True)

训练过程中，训练准确率单调上升、训练损失单调下降，但验证集准确率出现大幅波动，验证损失和准确率的波动情况可参考下方训练截图：

原定设置300轮训练，仅训练21轮就触发早停终止。初步判断为过拟合问题，调整dropout参数后波动问题仍未解决，需要定位问题根本原因。

内容的提问来源于stack exchange，提问作者Ahmad