自定义损失函数导致Keras时序回归模型MSE升高且输出偏移

模型结构 #

我正在训练一个用于时序回归的神经网络，模型结构如下：

####################################################################################################################
# Define ANN Model
# define two sets of inputs
acc   =  layers.Input(shape=(3,1,))
gyro  =  layers.Input(shape=(3,1,))

# the first branch operates on the first input
x = Conv1D(256, 1, activation='relu')(acc)
x = Conv1D(128, 1, activation='relu')(x)
x = Conv1D(128, 1, activation='relu')(x)
x = MaxPooling1D(pool_size=3)(x)
x = Model(inputs=acc, outputs=x)

# the second branch operates on the second input
y = Conv1D(256, 1, activation='relu')(gyro)
y = Conv1D(128, 1, activation='relu')(y)
y = Conv1D(128, 1, activation='relu')(y)
y = MaxPooling1D(pool_size=3)(y)
y = Model(inputs=gyro, outputs=y)


# combine the output of the two branches
combined =  layers.concatenate([x.output,  y.output])

# combined outputs
z = Bidirectional(LSTM(128, dropout=0.25, return_sequences=False,activation='tanh'))(combined)
z = Reshape((256,1),input_shape=(128,))
z = Bidirectional(LSTM(128, dropout=0.25, return_sequences=False,activation='tanh'))(combined)

#z = Dense(10, activation="relu")(z)
z = Flatten()(z)
z = Dense(4, activation="linear")(z)
model = Model(inputs=[x.input, y.input], outputs=z)
model.compile(loss=loss, optimizer = tf.keras.optimizers.Adam(),metrics=['mse'],run_eagerly=True)

自定义损失函数实现 #

基于相关论文实现了自定义损失函数，数学计算方式为：

y_pred = [w x y z]
y_true = [w1 x1 y1 z1]
error = 2 * acos(w*w1 + x*x1 + y*y1 + z*z1)

对应的代码实现：

def loss(y_true, y_pred):
    z = y_true * (y_pred )
    wtot = tf.reduce_sum(z,axis=1)
    error = 2*tf.math.acos(K.clip(tf.math.sqrt(wtot*wtot), -1.,1.))
    return error

训练异常现象 #

训练过程中出现损失值下降的同时MSE升高，且输出存在随训练轮次增加而逐渐增大的偏移。虽然模型未针对MSE优化，但从数学角度MSE应下降或收敛至1附近。

（注：橙色为目标/参考值，蓝色为网络输出，附1轮、10轮、50轮训练的结果图）

内容的提问来源于stack exchange，提问作者Arman Asgharpoor