Keras Tuner自定义Callback引发val_loss缺失报错的解决求助

问题现象 #

使用Keras Tuner进行超参数调优时，自定义Callback实现训练结束后的模型评估、参数保存、计时等功能，训练过程正常且结果能成功存储，但始终触发如下报错：

Search: Running Trial #1

Value             |Best Value So Far |Hyperparameter
64                |64                |filters_1
25                |25                |kernel_size_1
1                 |1                 |pool_size_1
2                 |2                 |strides_1
64                |64                |units

Epoch 1/3
6/6 [==============================] - 2s 215ms/step - loss: 0.0545 - accuracy: 0.1964 - val_loss: 0.0470 - val_accuracy: 0.2143
Epoch 2/3
6/6 [==============================] - 1s 120ms/step - loss: 0.0477 - accuracy: 0.4643 - val_loss: 0.0424 - val_accuracy: 0.4286
Epoch 3/3
6/6 [==============================] - 1s 126ms/step - loss: 0.0425 - accuracy: 0.5179 - val_loss: 0.0327 - val_accuracy: 0.5000
dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])
1/1 [==============================] - 0s 59ms/step - loss: 0.0178 - accuracy: 0.3000
1/1 [==============================] - 0s 172ms/step
INFO:tensorflow:Assets written to: ../models/hyperparametertuning1/with_background/100/Trial_0/model\assets
Results for trial 0 written to file.
dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])
Traceback (most recent call last):
  File "C:\Users\kle7ba\.conda\envs\ogs-ma\Lib\site-packages\keras_tuner\engine\base_tuner.py", line 270, in _try_run_and_update_trial
    self._run_and_update_trial(trial, *fit_args, **fit_kwargs)
  File "C:\Users\kle7ba\.conda\envs\ogs-ma\Lib\site-packages\keras_tuner\engine\base_tuner.py", line 257, in _run_and_update_trial
    self.oracle.update_trial(
  File "C:\Users\kle7ba\.conda\envs\ogs-ma\Lib\site-packages\keras_tuner\engine\oracle.py", line 107, in wrapped_func
    ret_val = func(*args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kle7ba\.conda\envs\ogs-ma\Lib\site-packages\keras_tuner\engine\oracle.py", line 364, in update_trial
    self._check_objective_found(metrics)
  File "C:\Users\kle7ba\.conda\envs\ogs-ma\Lib\site-packages\keras_tuner\engine\oracle.py", line 626, in _check_objective_found
    raise ValueError(
ValueError: Objective value missing in metrics reported to the Oracle, expected: ['val_loss'], found: dict_keys([])
Trial 1 Complete [00h 00m 10s]

移除自定义Callback仅保留早停回调时，报错消失。尝试改用Lambda Callback时，无法获取训练后的模型实例完成评估与预测。

相关代码 #

初始自定义Callback类 #

class CustomCallback(keras.callbacks.Callback):

    trial_number = 0
    results = pd.DataFrame(columns=['Trial-ID', 'loss', 'val_loss', 'test_loss', 'accuracy', 'val_accuracy', 'test_accuracy', 'training_time', 'prediction_time'])

    def __init__(self, X_test, y_test, max_trials, results_directory):
        self.X_test = X_test
        self.y_test = y_test
        self.max_trials = max_trials
        self.t_training_start = None
        self.results_directory = results_directory
        self.base_results_directory = results_directory
        self.history = {"loss": [], "val_loss": [], "accuracy": [], "val_accuracy": []}

    def on_train_begin(self, logs=None):
        # 记录训练开始时间
        self.t_training_start = time.time()

    def on_train_end(self, logs):
        # 训练结束后执行评估与保存
        print(logs.keys())
        self.results_directory = self.results_directory + '/Trial_' + str(CustomCallback.trial_number)
        model_directory = self.results_directory + '/model'
        figure_directory = self.results_directory + '/plots'
        prediction_directory = self.results_directory + '/predictions'

        # 计算训练时长
        t_training_end = time.time()
        training_time = t_training_end - self.t_training_start

        # 获取训练集与验证集指标
        loss = logs['loss']
        val_loss = logs['val_loss']
        accuracy = logs['accuracy']
        val_accuracy = logs['val_accuracy']

        # 测试集评估
        evaluation = self.model.evaluate(self.X_test, self.y_test)
        test_loss = evaluation[0]
        test_accuracy = evaluation[1]

        # 计算预测时长
        t_prediction_start = time.time()
        predictions = self.model.predict(self.X_test)
        t_prediction_end = time.time()
        prediction_time = t_prediction_end - t_prediction_start
        prediction_time_single = (t_prediction_end - t_prediction_start)/len(self.X_test)

        ...

随机搜索代码 #

# 定义超参数搜索器
tuner = kt.RandomSearch(
    build_model,
    objective='val_loss',
    max_trials=MAX_TRIALS,
    max_consecutive_failed_trials=MAX_TRIALS,
    overwrite=True,
    directory="tmp/",
    project_name='hyperparametertuning1'
)

# 执行超参数搜索
tuner.search(X_train, y_train, epochs=EPOCHS, batch_size=10, validation_split=0.2, callbacks=[early_stopping, CustomCallback(X_test=X_test, y_test=y_test, max_trials=MAX_TRIALS, results_directory=RESULTS_DIRECTORY)])

Lambda Callback尝试代码 #

# 实例化自定义Callback
custom_callback = Callbacks.CustomCallback(X_test=X_test, y_test=y_test, max_trials=MAX_TRIALS, results_directory=RESULTS_DIRECTORY)

lambda_timer_start = lambda logs : custom_callback.timer_start(logs)
lambda_set_model = lambda logs: custom_callback.result_saving(logs)

timer_start_callback = keras.callbacks.LambdaCallback(on_train_begin=lambda_timer_start)
result_saving_callback = keras.callbacks.LambdaCallback(on_train_end=lambda_set_model)

修改后的Callback类 #

class CustomCallback(keras.callbacks.Callback):

    trial_number = 0
    results = pd.DataFrame(columns=['Trial-ID', 'loss', 'val_loss', 'test_loss', 'accuracy', 'val_accuracy', 'test_accuracy', 'training_time', 'prediction_time'])

    def __init__(self, X_test, y_test, max_trials, results_directory):
        self.X_test = X_test
        self.y_test = y_test
        self.max_trials = max_trials
        self.t_training_start = None
        self.results_directory = results_directory
        self.base_results_directory = results_directory
        self.history = {"loss": [], "val_loss": [], "accuracy": [], "val_accuracy": []}

    def timer_start(self, logs=None):
        # 记录训练开始时间
        self.t_training_start = time.time()

    def result_saving(self, logs=None):
        # 训练结束后执行评估与保存
        self.results_directory = self.results_directory + '/Trial_' + str(CustomCallback.trial_number)
        model_directory = self.results_directory + '/model'
        figure_directory = self.results_directory + '/plots'
        prediction_directory = self.results_directory + '/predictions'

        # 计算训练时长
        t_training_end = time.time()
        training_time = t_training_end - self.t_training_start

        # 获取训练集与验证集指标
        loss = logs.get('loss')
        val_loss = logs.get('val_loss')
        accuracy = logs.get('accuracy')
        val_accuracy = logs.get('val_accuracy')

        # 测试集评估
        evaluation = self.model.evaluate(self.X_test, self.y_test)
        test_loss = evaluation[0]
        test_accuracy = evaluation[1]

        # 计算预测时长
        t_prediction_start = time.time()
        predictions = self.model.predict(self.X_test)
        t_prediction_end = time.time()
        prediction_time = t_prediction_end - t_prediction_start
        prediction_time_single = (t_prediction_end - t_prediction_start)/len(self.X_test)

        ...

问题原因与解决方法 #

核心原因 #

自定义Callback的on_train_end方法没有返回原始的logs字典，导致Keras Tuner的Oracle组件无法获取到指定的目标指标val_loss。

解决步骤 #

1. 初始版本Callback修复：在on_train_end方法末尾添加return logs，确保原始指标字典被传递给后续流程：

def on_train_end(self, logs):
    # 你的原有代码...
    
    # 最后添加此行
    return logs

2. 修改后版本Callback修复：如果使用拆分的result_saving方法，同样需要在方法末尾返回logs：

def result_saving(self, logs=None):
    # 你的原有代码...
    
    # 最后添加此行
    return logs

3. 额外注意：不要在Callback中修改logs字典的原有结构，确保val_loss等目标键值对完整保留；同时全局变量trial_number需要在每个trial结束后递增，避免路径重复。

内容的提问来源于stack exchange，提问作者Lukas