如何在Keras Tuner中纳入LeakyReLU等高级激活层作为超参选项

我有一段用Keras Tuner构建单隐藏层神经网络的Python代码，目前只支持relu和elu两种激活函数。但LeakyReLU、PReLU这类高级激活层的调用方式和普通激活函数不一样，想把这4种激活层都加入超参数搜索选项，该怎么改代码？

原代码如下：

def build_model(hp):
  
  # Initialize sequential
  model = keras.Sequential()

  # Tune the number of units in the first Dense layer
  hp_units = hp.Int('units', min_value=25, max_value=250, step=25)
  
  model.add(keras.layers.Dense(units=hp_units, input_shape = (train_pca_scaled.shape[1],),
                               kernel_regularizer=keras.regularizers.L1(l1 = hp.Choice('L1_value', [1e-2, 1e-3, 1e-4]))))
  
    
  # Batch Normalization (before activation)
  model.add(keras.layers.BatchNormalization())
  
  # Activation
  activation = hp.Choice("activation", ["relu", "elu"])
  model.add(keras.layers.Activation(activation))
  
  # Tune dropout rate
  hp_dropout = hp.Float('dropout_rate', min_value=0, max_value=0.5, step=0.1)
  model.add(keras.layers.Dropout(rate = hp_dropout))
        
  # Final output layer
  model.add(keras.layers.Dense(units = 1))
      
  # Compile
  lr = hp.Choice("learning_rate", values=[1e-2, 1e-3, 1e-4])
  model.compile(optimizer = keras.optimizers.Adam(learning_rate= lr),
                   loss = keras.losses.MeanSquaredError(), 
                   metrics= [keras.metrics.RootMeanSquaredError()])

  return model

核心思路是通过条件判断区分普通激活函数和高级激活层的调用方式：普通激活函数（relu/elu）沿用Activation层，高级激活层（LeakyReLU/PReLU）直接实例化对应的层类，同时扩展超参数选项。

修改后的完整代码 #

def build_model(hp):
  
  # Initialize sequential
  model = keras.Sequential()

  # Tune the number of units in the first Dense layer
  hp_units = hp.Int('units', min_value=25, max_value=250, step=25)
  
  model.add(keras.layers.Dense(units=hp_units, input_shape = (train_pca_scaled.shape[1],),
                               kernel_regularizer=keras.regularizers.L1(l1 = hp.Choice('L1_value', [1e-2, 1e-3, 1e-4]))))
  
    
  # Batch Normalization (before activation)
  model.add(keras.layers.BatchNormalization())
  
  # Activation: 支持4种激活层
  activation_type = hp.Choice("activation", ["relu", "elu", "leaky_relu", "prelu"])
  
  if activation_type in ["relu", "elu"]:
      model.add(keras.layers.Activation(activation_type))
  elif activation_type == "leaky_relu":
      # 可选：将alpha设为超参数搜索项，固定值的话直接写alpha=0.2即可
      alpha = hp.Float('leaky_relu_alpha', min_value=0.01, max_value=0.3, step=0.05)
      model.add(keras.layers.LeakyReLU(alpha=alpha))
  elif activation_type == "prelu":
      # PReLU的alpha是可训练参数，默认初始值为0.25，无需手动指定
      model.add(keras.layers.PReLU())
  
  # Tune dropout rate
  hp_dropout = hp.Float('dropout_rate', min_value=0, max_value=0.5, step=0.1)
  model.add(keras.layers.Dropout(rate = hp_dropout))
        
  # Final output layer
  model.add(keras.layers.Dense(units = 1))
      
  # Compile
  lr = hp.Choice("learning_rate", values=[1e-2, 1e-3, 1e-4])
  model.compile(optimizer = keras.optimizers.Adam(learning_rate= lr),
                   loss = keras.losses.MeanSquaredError(), 
                   metrics= [keras.metrics.RootMeanSquaredError()])

  return model

额外说明 #

• LeakyReLU的alpha值如果不需要调参，可直接固定为常见值（比如0.2），去掉对应的hp.Float配置即可
• PReLU的alpha是模型训练时会自动更新的可训练参数，默认初始值为0.25，一般无需手动调整

内容的提问来源于stack exchange，提问作者pdhami