在Keras-Tuner中使用F1 Score作为优化指标报错的解决方案问询

尝试用Keras-Tuner优化二分类图像模型的F1分数，由于Keras移除了默认F1指标，改用TensorFlow Addons的F1Score类，但运行时触发KeyError，报错显示Keras-Tuner无法识别该指标。

原实现代码 #

def model_builder(hp):
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Rescaling(scale=255))
    model.add(tf.keras.layers.TimeDistributed(net))
    model.add(tf.keras.layers.Dense(units=hp.Int(
        'units', min_value=32, max_value=512, step=32), activation='relu'))
    model.add(tf.keras.layers.GlobalAveragePooling3D())
    model.add(tf.keras.layers.Dense(1, activation='sigmoid'))

    custom_optimizer = keras.optimizers.Adam(
        learning_rate=hp.Choice('learning_rate', values=[1e-2, 1e-3, 1e-4]),
        beta_1=hp.Choice('beta_1', values=[0.9, 0.99, 0.999]),
        beta_2=hp.Choice('beta_2', values=[0.999, 0.9999]),
        epsilon=hp.Float('epsilon', min_value=1e-10, max_value=1e-7)
    )

    # Running with SGD optimizer
    model.compile(optimizer='sgd',
                  loss=keras.losses.binary_crossentropy, metrics=tfa.metrics.F1Score(num_classes=1, average='macro',threshold=0.5))
    
    
    return model

# Initialize the tuner
tuner = RandomSearch(
    model_builder,
    objective=Objective(tfa.metrics.F1Score(num_classes=1, average='macro',threshold=0.5), direction=max),
    max_trials=10,
    directory='test_directory/logs'
)

# Start the tuning process
combined = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)]
tuner.search(train_ds, epochs=10, validation_data=val_ds, callbacks=combined)

报错信息 #

RuntimeError                              Traceback (most recent call last)
 3 combined = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)]
      5 # Start the tuning process
----> 6 tuner.search(train_ds, epochs=10, validation_data=(
      7     val_ds), callbacks=combined)

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\keras_tuner\src\engine\base_tuner.py:234, in BaseTuner.search(self, *fit_args, **fit_kwargs)
    232     self.on_trial_begin(trial)
    233     self._try_run_and_update_trial(trial, *fit_args, **fit_kwargs)
---> 234     self.on_trial_end(trial)
    235 self.on_search_end()

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\keras_tuner\src\engine\base_tuner.py:338, in BaseTuner.on_trial_end(self, trial)
    332 def on_trial_end(self, trial):
    333     """Called at the end of a trial.
    334 
    335     Args:
    336         trial: A `Trial` instance.
    337     """
---> 338     self.oracle.end_trial(trial)
    339     self.save()

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\keras_tuner\src\engine\oracle.py:108, in synchronized.<locals>.wrapped_func(*args, **kwargs)
...
  File "C:\Users\name_here\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\keras_tuner\src\engine\objective.py", line 59, in get_value
    return logs[self.name]
           ~~~~^^^^^^^^^^^
KeyError: <tensorflow_addons.metrics.f_scores.F1Score object at 0x000001C8709D6710>

问题原因 #

1. Keras-Tuner的Objective参数需要传入指标名称字符串，而非指标实例，原代码直接传入F1Score实例导致无法匹配日志中的键。
2. 二分类场景下使用average='macro'搭配num_classes=1不符合逻辑，应该使用average='binary'更适配二分类任务。
3. 未给F1Score设置明确的name，导致日志中生成的指标键不规范，无法被Keras-Tuner识别。

解决方法 #

1. 初始化F1Score时指定name参数，确保日志中能生成对应的键。
2. 在Objective中使用该名称字符串，而非指标实例。
3. 调整F1Score的参数，二分类场景下设置average='binary'，无需指定num_classes（或设置num_classes=2）。

修改后的完整代码 #

def model_builder(hp):
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Rescaling(scale=255))
    model.add(tf.keras.layers.TimeDistributed(net))
    model.add(tf.keras.layers.Dense(units=hp.Int(
        'units', min_value=32, max_value=512, step=32), activation='relu'))
    model.add(tf.keras.layers.GlobalAveragePooling3D())
    model.add(tf.keras.layers.Dense(1, activation='sigmoid'))

    custom_optimizer = keras.optimizers.Adam(
        learning_rate=hp.Choice('learning_rate', values=[1e-2, 1e-3, 1e-4]),
        beta_1=hp.Choice('beta_1', values=[0.9, 0.99, 0.999]),
        beta_2=hp.Choice('beta_2', values=[0.999, 0.9999]),
        epsilon=hp.Float('epsilon', min_value=1e-10, max_value=1e-7)
    )

    # 初始化带名称的F1指标，适配二分类
    f1_metric = tfa.metrics.F1Score(average='binary', threshold=0.5, name='f1_score')
    
    # Running with SGD optimizer
    model.compile(optimizer='sgd',
                  loss=keras.losses.binary_crossentropy, metrics=[f1_metric])
    
    
    return model

# Initialize the tuner
tuner = RandomSearch(
    model_builder,
    # 使用指标名称字符串，而非实例，验证集指标需加val_前缀
    objective=Objective('val_f1_score', direction='max'),
    max_trials=10,
    directory='test_directory/logs'
)

# Start the tuning process
combined = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)]
tuner.search(train_ds, epochs=10, validation_data=val_ds, callbacks=combined)

额外说明 #

• 监控验证集指标时，需要在名称前加val_前缀，这是Keras默认的验证指标命名规则。
• 若坚持使用num_classes=1，需将average设置为'macro'，但二分类场景下binary更合理，计算的是正类的F1分数。

内容的提问来源于stack exchange，提问作者user22931525