Keras中基于其他输出为模型输出损失添加正则化的实现

方案1：利用函数式API的add_loss方法 #

这种方法无需拆分损失列表，直接在模型构建阶段将包含交叉正则的两个损失加入模型，严格保证各分支损失的训练逻辑。

步骤： #

1. 构建函数式模型
   假设三个输入为input1、input2、input3，通过分支网络得到两个输出v_output、q_output：

   from tensorflow.keras.layers import Input, Dense, Concatenate
   from tensorflow.keras.models import Model
   import tensorflow.keras.backend as K
   
   # 定义输入
   input1 = Input(shape=(...))
   input2 = Input(shape=(...))
   input3 = Input(shape=(...))
   
   # 共享基础网络（可根据需求调整为独立分支）
   shared = Concatenate()([input1, input2, input3])
   shared = Dense(64, activation='relu')(shared)
   
   # V分支输出
   v_branch = Dense(32, activation='relu')(shared)
   v_output = Dense(10, name='v_output')(v_branch)
   
   # Q分支输出
   q_branch = Dense(32, activation='relu')(shared)
   q_output = Dense(10, name='q_output')(q_branch)
   

2. 引入目标输入并计算损失
   将y_v、y_q作为额外输入传入模型，才能在损失中计算对应MSE，同时定义交叉正则项：

   # 定义目标值输入
   target_v = Input(shape=(10,), name='target_v')
   target_q = Input(shape=(10,), name='target_q')
   
   # 正则项权重参数
   lambda_ = 0.01
   
   # 定义正则函数f(v,q)（示例：最小化v与q的点积绝对值均值）
   def f(v, q):
       dot_product = K.batch_dot(v, q, axes=1)  # 逐样本计算点积
       return K.mean(K.abs(dot_product))
   
   # 计算带正则的两个损失
   L_V = K.mean(K.square(v_output - target_v)) + lambda_ * f(v_output, q_output)
   L_Q = K.mean(K.square(q_output - target_q)) + lambda_ * f(v_output, q_output)
   
   # 初始化模型并添加损失
   model = Model(inputs=[input1, input2, input3, target_v, target_q], outputs=[v_output, q_output])
   model.add_loss(L_V)
   model.add_loss(L_Q)
   

3. 编译与训练
   编译时无需指定loss参数，直接传入优化器即可：

   model.compile(optimizer='adam')
   
   # 训练时输入包含3个模型输入+2个目标值，输出参数可设为None
   model.fit(
       x=[x1_train, x2_train, x3_train, y_v_train, y_q_train],
       y=None,
       epochs=50,
       batch_size=32
   )
   

方案2：自定义训练循环（更灵活可控） #

如果需要精细控制训练流程，推荐用TensorFlow自定义训练循环，直接计算损失与梯度：

步骤： #

1. 构建基础模型
   无需额外目标输入，直接定义输入和双输出：

   input1 = Input(shape=(...))
   input2 = Input(shape=(...))
   input3 = Input(shape=(...))
   
   shared = Concatenate()([input1, input2, input3])
   shared = Dense(64, activation='relu')(shared)
   
   v_branch = Dense(32, activation='relu')(shared)
   v_output = Dense(10, name='v_output')(v_branch)
   
   q_branch = Dense(32, activation='relu')(shared)
   q_output = Dense(10, name='q_output')(q_branch)
   
   model = Model(inputs=[input1, input2, input3], outputs=[v_output, q_output])
   

2. 定义优化器与损失逻辑

   import tensorflow as tf
   
   optimizer = tf.keras.optimizers.Adam()
   lambda_ = 0.01
   
   def compute_loss(v_pred, q_pred, y_v, y_q):
       # 计算各分支MSE损失
       mse_v = tf.reduce_mean(tf.square(v_pred - y_v))
       mse_q = tf.reduce_mean(tf.square(q_pred - y_q))
       # 计算交叉正则项
       dot_product = tf.reduce_sum(v_pred * q_pred, axis=1)
       reg_term = tf.reduce_mean(tf.abs(dot_product))
       # 总损失为两个带正则的损失之和
       total_loss = (mse_v + lambda_ * reg_term) + (mse_q + lambda_ * reg_term)
       return total_loss, mse_v, mse_q, reg_term
   

3. 编写训练循环

   @tf.function
   def train_step(inputs, targets):
       x1, x2, x3 = inputs
       y_v, y_q = targets
   
       with tf.GradientTape() as tape:
           v_pred, q_pred = model([x1, x2, x3], training=True)
           total_loss, mse_v, mse_q, reg_term = compute_loss(v_pred, q_pred, y_v, y_q)
   
       # 计算梯度并更新模型参数
       gradients = tape.gradient(total_loss, model.trainable_variables)
       optimizer.apply_gradients(zip(gradients, model.trainable_variables))
   
       return {'total_loss': total_loss, 'mse_v': mse_v, 'mse_q': mse_q, 'reg_term': reg_term}
   
   # 训练流程
   epochs = 50
   batch_size = 32
   
   # 构建训练数据集（用tf.data.Dataset提升效率）
   train_dataset = tf.data.Dataset.from_tensor_slices(
       ([x1_train, x2_train, x3_train], [y_v_train, y_q_train])
   ).shuffle(1000).batch(batch_size)
   
   for epoch in range(epochs):
       print(f"Epoch {epoch+1}/{epochs}")
       for batch_inputs, batch_targets in train_dataset:
           metrics = train_step(batch_inputs, batch_targets)
       
       # 打印epoch指标
       print(f"Total Loss: {metrics['total_loss'].numpy():.4f}, MSE V: {metrics['mse_v'].numpy():.4f}, MSE Q: {metrics['mse_q'].numpy():.4f}, Reg Term: {metrics['reg_term'].numpy():.4f}")
   

原拼接输出方案失效原因 #

你之前将V和Q拼接为单一输出、返回L_v + L_q + L_regu的方式，会导致V分支参数被Q的MSE损失反向更新，Q分支参数被V的MSE损失反向更新，完全违背了两个分支的训练目标，最终导致模型无法收敛。上面的两种方案都严格保证L_V仅作用于V分支、L_Q仅作用于Q分支，同时通过共享正则项实现两个输出的约束。

内容的提问来源于stack exchange，提问作者Alireza