TensorFlow 2中CNN模型使用零一损失函数训练报错排查

我正在完成一项大学考试项目，需开发用于图像二分类的卷积神经网络（CNN），项目明确要求使用zero-one-loss（零一损失函数）。在搭建基线模型以进行后续超参数调优时，使用自定义零一损失函数训练模型出现报错。

模型定义代码 #

def baseline_model_v0():
  img_shape = (IMG_HEIGHT, IMG_WIDTH, CHANNELS)
  model = Sequential()
  model.add(Rescaling(1./255, input_shape=img_shape))
  
  model.add(Conv2D(filters=64, kernel_size=(3, 3), padding="same", activation="relu"))
  model.add(keras.layers.MaxPool2D(padding="same"))
  model.add(Conv2D(filters=128, kernel_size=(3, 3), padding="same", activation="relu"))
  model.add(keras.layers.MaxPool2D(padding="same"))
  model.add(Conv2D(filters=256, kernel_size=(3, 3), padding="same", activation="relu"))
  model.add(keras.layers.MaxPool2D(padding="same"))

  model.add(Flatten())

  model.add(Dense(units=512, activation="relu"))
  model.add(Dense(units=512, activation="relu"))
  model.add(Dense(units=512, activation="relu"))
  model.add(Dense(units=1, activation="sigmoid"))

  model.compile(optimizer=keras.optimizers.Adam(learning_rate=0.01),
                loss=zero_one_loss,
                metrics=METRICS)

  return model

自定义零一损失函数 #

def zero_one_loss(y_true, y_pred):
  res = 1.0 if y_true != y_pred else 0.0
  return res

训练时报错信息 #

model = baseline_model_v0()
history_v0 = model.fit(train_ds, validation_data=test_ds, epochs=30)

Epoch 1/30
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-18-3c67809f14c1> in <module>
      1 model = baseline_model_v0()
----> 2 history_v0 = model.fit(train_ds, validation_data=test_ds, epochs=30)

1 frames
/usr/local/lib/python3.7/dist-packages/keras/engine/training.py in tf__train_function(iterator)
     13                 try:
     14                     do_return = True
---&gt; 15                     retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
     16                 except:
     17                     do_return = False

ValueError: in user code:

    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1160, in train_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1146, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1135, in run_step  **
        outputs = model.train_step(data)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 997, in train_step
        self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
    File "/usr/local/lib/python3.7/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 579, in minimize
        return self.apply_gradients(grads_and_vars, name=name)
    File "/usr/local/lib/python3.7/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 689, in apply_gradients
        grads_and_vars = optimizer_utils.filter_empty_gradients(grads_and_vars)
    File "/usr/local/lib/python3.7/dist-packages/keras/optimizers/optimizer_v2/utils.py", line 78, in filter_empty_gradients
        f"No gradients provided for any variable: {variable}. "

    ValueError: No gradients provided for any variable: (['conv2d_3/kernel:0', 'conv2d_3/bias:0', 'conv2d_4/kernel:0', 'conv2d_4/bias:0', 'conv2d_5/kernel:0', 'conv2d_5/bias:0', 'dense_4/kernel:0', 'dense_4/bias:0', 'dense_5/kernel:0', 'dense_5/bias:0', 'dense_6/kernel:0', 'dense_6/bias:0', 'dense_7/kernel:0', 'dense_7/bias:0'],). Provided `grads_and_vars` is ((None, <tf.Variable 'conv2d_3/kernel:0' shape=(3, 3, 3, 64) dtype=float32>), (None, <tf.Variable 'conv2d_3/bias:0' shape=(64,) dtype=float32>), (None, <tf.Variable 'conv2d_4/kernel:0' shape=(3, 3, 64, 128) dtype=float32>), (None, <tf.Variable 'conv2d_4/bias:0' shape=(128,) dtype=float32>), (None, <tf.Variable 'conv2d_5/kernel:0' shape=(3, 3, 128, 256) dtype=float32>), (None, <tf.Variable 'conv2d_5/bias:0' shape=(256,) dtype=float32>), (None, <tf.Variable 'dense_4/kernel:0' shape=(262144, 512) dtype=float32>), (None, <tf.Variable 'dense_4/bias:0' shape=(512,) dtype=float32>), (None, <tf.Variable 'dense_5/kernel:0' shape=(512, 512) dtype=float32>), (None, <tf.Variable 'dense_5/bias:0' shape=(512,) dtype=float32>), (None, <tf.Variable 'dense_6/kernel:0' shape=(512, 512) dtype=float32>), (None, <tf.Variable 'dense_6/bias:0' shape=(512,) dtype=float32>), (None, <tf.Variable 'dense_7/kernel:0' shape=(512, 1) dtype=float32>), (None, <tf.Variable 'dense_7/bias:0' shape=(1,) dtype=float32>)).

错误原因 #

1. 非可微分操作导致梯度中断：你写的zero_one_loss用了Python原生的if-else和!=比较，这些不是TensorFlow的图操作，无法被自动微分机制追踪，导致反向传播时无法计算梯度，最终所有参数的梯度都为None。
2. 零一损失本身的特性：零一损失是离散的阶跃函数，大部分点的导数为0或不存在，根本不适合用于梯度下降训练——梯度下降需要损失函数是连续且可微分的。

解决方案 #

方案1：训练用交叉熵损失，评估用零一损失（推荐） #

因为项目要求用零一损失，通常是指评估指标而非训练损失。实际训练时用二元交叉熵（它是零一损失的光滑上界，可微分，适合梯度下降），仅在评估阶段计算零一损失作为指标。

修改代码如下：

import tensorflow as tf

# 自定义零一损失作为评估指标
def zero_one_metric(y_true, y_pred):
    # 先把sigmoid输出转为0/1预测
    y_pred = tf.cast(y_pred > 0.5, tf.float32)
    # 计算错误样本占比（零一损失的均值）
    return tf.reduce_mean(tf.cast(tf.not_equal(y_true, y_pred), tf.float32))

def baseline_model_v0():
  img_shape = (IMG_HEIGHT, IMG_WIDTH, CHANNELS)
  model = Sequential()
  model.add(Rescaling(1./255, input_shape=img_shape))
  
  model.add(Conv2D(filters=64, kernel_size=(3, 3), padding="same", activation="relu"))
  model.add(keras.layers.MaxPool2D(padding="same"))
  model.add(Conv2D(filters=128, kernel_size=(3, 3), padding="same", activation="relu"))
  model.add(keras.layers.MaxPool2D(padding="same"))
  model.add(Conv2D(filters=256, kernel_size=(3, 3), padding="same", activation="relu"))
  model.add(keras.layers.MaxPool2D(padding="same"))

  model.add(Flatten())

  model.add(Dense(units=512, activation="relu"))
  model.add(Dense(units=512, activation="relu"))
  model.add(Dense(units=512, activation="relu"))
  model.add(Dense(units=1, activation="sigmoid"))

  # 训练用二元交叉熵，评估加入零一损失指标
  model.compile(optimizer=keras.optimizers.Adam(learning_rate=0.01),
                loss=keras.losses.BinaryCrossentropy(),
                metrics=[METRICS, zero_one_metric])

  return model

方案2：使用可微分的零一损失近似（如果必须用类似损失训练） #

如果项目强制要求用零一损失类的函数训练，可以用平滑零一损失，它是连续可微分的近似：

import tensorflow as tf

def smooth_zero_one_loss(y_true, y_pred):
    # y_pred是sigmoid输出，用平滑函数近似零一损失
    threshold = 0.5
    diff = tf.abs(y_true - y_pred)
    return tf.where(diff < threshold, diff / threshold, 1.0)

# 编译时使用这个平滑损失
model.compile(optimizer=keras.optimizers.Adam(learning_rate=0.01),
              loss=smooth_zero_one_loss,
              metrics=[METRICS])

内容的提问来源于stack exchange，提问作者IfLoveWasBornToDie92