Keras中应用Cyclical Learning Rate出现TypeError的解决咨询

在Keras中测试循环学习率（Cyclical Learning Rate）时，第一个epoch结束触发TypeError: unsupported operand type(s) for *: 'CyclicalLearningRate' and 'int'错误，完整报错栈如下：

Epoch 1/10000
hist = model.fit_generator(generator=training_generator,validation_data = calibration_generator, epochs = n_epoch,
510/510 [==============================] - ETA: 0s - loss: 0.0111 - mae: 0.0772  Traceback (most recent call last):

File "H:\my_data\from drive C my PC Spet_2022\Code\Phase_2_Cyclic_Learning.py", line 303, in 
hist = model.fit_generator(generator=training_generator,validation_data calibration_generator, epochs = n_epoch,

File "C:\Users\bluesky\AppData\Roaming\Python\Python38\site-packages\keras\engine\training.py", line 2507, in fit_generator
return self.fit(

File "C:\Users\bluesky\AppData\Roaming\Python\Python38\site-packages\keras\utils\traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None

File "C:\Users\bluesky\AppData\Roaming\Python\Python38\site-packages\keras\utils\generic_utils.py", line 965, in update
self._values[k] = [v * value_base, value_base]

TypeError: unsupported operand type(s) for *: 'CyclicalLearningRate' and 'int'

相关代码如下：

pip install tensorflow-addons
from tensorflow_addons.optimizers import CyclicalLearningRate


model = Sequential()


input_dim = len(input_cols_idx)
batch_size = batch_size_gen


model.add(Dense(units=hidden_layer1_neurons, input_dim= input_dim,
        kernel_initializer = initializers.RandomNormal(mean=0.0, stddev=0.05)))


model.add(Activation(activation_1)) 
model.add(Dense(units=1))
model.add(Activation(activation_2)) 



# Create Cyclic Learning Rate and Callbacks

cyclical_learning_rate = CyclicalLearningRate(
initial_learning_rate=0.000008,
maximal_learning_rate=0.001,
step_size=2360,
scale_fn=lambda x: 1 / (2.0 ** (x - 1)),
scale_mode='cycle')

model.compile(loss='mean_squared_error', 
 optimizer=Adam(learning_rate=cyclical_learning_rate), metrics=['mae'])

checkpointer = callbacks.ModelCheckpoint(filepath=("%s/%s.h5" % (model_dir,model_name)), 
monitor='val_loss', mode='min', save_best_only=True) 

# val_loss                                  
reduce = callbacks.ReduceLROnPlateau(monitor='val_mae',factor=0.25, mode='min',
                             patience = lr_reduce_patience,  epsilon=epsilon,
                             cooldown=0,  min_lr= min_lr)    # Later versions of Keras 
uses min_delta inplace of epsilon
earlystop = callbacks.EarlyStopping(monitor='val_mae', min_delta = earlystop_min_delta, 
                            patience=earlystop_patience, verbose=0, mode='min')

csv_logger = CSVLogger('%s/%s_log.csv'%(model_dir, model_name), append=True, 
  separator=';')

训练代码：

hist = model.fit_generator(generator=training_generator,validation_data = 
  calibration_generator, epochs = n_epoch,
                use_multiprocessing=False, callbacks = [reduce, earlystop,checkpointer, 
csv_logger]) #TensorBoard(log_dir='./logs')])
               # workers=6)    ,plot_losses        
cost=pd.DataFrame(hist.history)

错误原因 #

报错核心是**ReduceLROnPlateau回调与CyclicalLearningRate冲突**：

• ReduceLROnPlateau会在验证指标停滞时尝试对当前学习率执行乘法操作（乘以factor参数）
• 但CyclicalLearningRate是一个自定义学习率调度对象，并非数值类型，无法直接和整数/浮点数做乘法运算，因此触发类型错误。

解决步骤 #

1. 移除ReduceLROnPlateau回调
   循环学习率本身已经实现了动态调整学习率的策略，不需要再叠加学习率衰减回调，否则会干扰循环逻辑。修改后的回调列表如下：

   callbacks = [earlystop, checkpointer, csv_logger]
   

2. 替换过时的fit_generator方法
   新版本Keras中fit_generator已被弃用，直接使用model.fit即可支持生成器输入：

   hist = model.fit(
       generator=training_generator,
       validation_data=calibration_generator,
       epochs=n_epoch,
       use_multiprocessing=False,
       callbacks=[earlystop, checkpointer, csv_logger]
   )
   

3. 可选：自定义学习率调度逻辑（不推荐）
   如果一定要结合学习率衰减，需要修改CyclicalLearningRate的scale_fn，将衰减逻辑整合到循环策略中，例如：

   def custom_scale_fn(x):
       # 同时实现循环缩放和衰减
       cycle_scale = 1 / (2.0 ** (x - 1))
       decay_scale = 0.99 ** x  # 额外的衰减因子
       return cycle_scale * decay_scale
   
   cyclical_learning_rate = CyclicalLearningRate(
       initial_learning_rate=0.000008,
       maximal_learning_rate=0.001,
       step_size=2360,
       scale_fn=custom_scale_fn,
       scale_mode='cycle'
   )
   

   但这种方式会让学习率逻辑变得复杂，建议优先采用第一种方案。

内容的提问来源于stack exchange，提问作者jam_pyt