Keras多输入模型批量训练报错：无法转换NumPy数组为Tensor

我在训练一个多输入神经网络，输入是[state, other_inputs]，其中state形状为(40, 40, 1)，other_inputs形状为(3)。单独预测没问题，但批量训练时抛出异常。

模型定义 #

def create_model(self, input_shape=(40, 40, 1)):
    input_map = Layers.Input(shape=input_shape)
    input_other = Layers.Input(shape=(3))

    x = Layers.Conv2D(64, 5, strides=1, padding="same",
                      activation=tf.keras.layers.LeakyReLU(alpha=0.1), use_bias=True, bias_initializer='zeros')(
        input_map)
    x = Layers.Conv2D(64, 5, strides=1, padding="same", use_bias=True,
                      activation=tf.keras.layers.LeakyReLU(alpha=0.1),
                      bias_initializer='zeros')(x)

    x = Layers.Conv2D(1, 5, strides=1, padding="same", use_bias=True,
                      bias_initializer='zeros')(x)
    x = Layers.Flatten()(x)
    x = Layers.Dense(64, activation=tf.keras.layers.LeakyReLU(alpha=0.1))(x)
    x = Layers.Dense(64, activation=tf.keras.layers.LeakyReLU(alpha=0.1))(x)

    x = Layers.Dense(5, activation=tf.keras.layers.LeakyReLU(alpha=0.1))(x)

    concatted = tf.keras.layers.Concatenate()([x, input_other])
    output = Layers.Dense(2, activation='sigmoid')(concatted)

    model = Model(inputs=[input_map, input_other], outputs=output)
    model.compile(loss="binary_crossentropy",
                  optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001),
                  metrics=["mae"])
    model.summary()
    return model

训练代码 #

def train(self, memory, target_model, batch_size, learning_rate, discount_factor, episode,  rarl_wins):
    X_train = []
    y_train = []
    y_pred = []
    self.gamma = discount_factor
    mini_batch = random.sample(memory, batch_size)

    states = np.array([transition[0] for transition in mini_batch])
    other_inputs = np.array([transition[1][0] for transition in mini_batch])
    qs_list = self.model.predict([states, other_inputs], verbose=0)

    next_states = np.array([transition[4] for transition in mini_batch])
    next_other_inputs = np.array([transition[5][0] for transition in mini_batch])
    next_qs_list = target_model.predict([next_states, next_other_inputs], verbose=0)

    for index, (state, other_inputs, action, reward, next_state, next_other_inputs, complete, expert_qs) in enumerate(mini_batch):
        if not complete:
            target = reward + self.gamma * np.amax(next_qs_list[index])
        else:
            target = reward

        qs = qs_list[index]
        qs[action] = (1 - learning_rate) * qs[action] + learning_rate * target

        X_train.append(np.array([state, other_inputs[0]]))
        y_train.append(qs)

    X_train = np.array(X_train)
    y_train = np.array(y_train)

    history= self.model.fit(X_train, y_train, batch_size=int(batch_size/5),shuffle=True)

    return self.model, history

错误信息 #

Traceback (most recent call last):
  File "\sas\train.py", line 397, in <module>
    main()
  File "\sas\train.py", line 236, in main
    _, history = agent.train(replay_memory, target_model=target_agent.model,
  File "\sas\ddqn\PAgent.py", line 226, in train
    history= self.model.fit(X_train, y_train, batch_size=int(batch_size/5),shuffle=True)
  File "\lib\site-packages\keras\utils\traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "\lib\site-packages\tensorflow\python\framework\constant_op.py", line 102, in convert_to_eager_tensor
    return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type numpy.ndarray).

问题原因与修复方案 #

问题根源 #

当前构建X_train的方式错误：你把每个样本的state和other_inputs打包成一个数组存入X_train，最终得到的是形状为(batch_size, 2)的数组，其中每个元素是不同形状的子数组（一个是(40,40,1)，一个是(3)）。而Keras多输入模型要求传入两个独立的数组/张量，分别对应两个输入层，而非一个包含子数组的数组。

修复步骤 #

1. 拆分输入数据：分别收集state和other_inputs，不要打包成一个数组：

   def train(self, memory, target_model, batch_size, learning_rate, discount_factor, episode,  rarl_wins):
       # 替换原X_train，改为两个独立列表
       X_train_state = []
       X_train_other = []
       y_train = []
       self.gamma = discount_factor
       mini_batch = random.sample(memory, batch_size)
   
       # ... 原有预测qs_list和next_qs_list的代码 ...
   
       for index, (state, other_inputs, action, reward, next_state, next_other_inputs, complete, expert_qs) in enumerate(mini_batch):
           # ... 原有计算target和qs的代码 ...
           
           # 分别添加到对应列表
           X_train_state.append(state)
           X_train_other.append(other_inputs[0])  # 确认other_inputs[0]是形状(3)的数组
           y_train.append(qs)
   
       # 转为NumPy数组
       X_train_state = np.array(X_train_state)
       X_train_other = np.array(X_train_other)
       y_train = np.array(y_train)
   

2. 修改fit调用方式：传入两个输入数组组成的列表：

   history = self.model.fit([X_train_state, X_train_other], y_train, batch_size=int(batch_size/5), shuffle=True)
   

额外验证点 #

• 确认X_train_state的形状为(batch_size, 40, 40, 1)，X_train_other形状为(batch_size, 3)，与模型输入层形状匹配。
• 检查other_inputs[0]确实是形状为(3)的数组，若不是需调整数据处理逻辑。

内容的提问来源于stack exchange，提问作者E.T.Tuna