Keras训练报错ValueError：无法找到数据适配器，如何解决？

搭建3D残差网络模型后，执行model.fit时出现数据适配错误，报错信息如下：

Traceback (most recent call last):
  File "/usr/lib64/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/miran045/reine097/projects/ResNet34/venv/lib64/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 2833, in variable_creator_scope
    yield
  File "/home/miran045/reine097/projects/ResNet34/venv/lib/python3.6/site-packages/keras/engine/training.py", line 1148, in fit
    steps_per_execution=self._steps_per_execution)
  File "/home/miran045/reine097/projects/ResNet34/venv/lib/python3.6/site-packages/keras/engine/data_adapter.py", line 1383, in get_data_handler
    return DataHandler(*args, **kwargs)
  File "/home/miran045/reine097/projects/ResNet34/venv/lib/python3.6/site-packages/keras/engine/data_adapter.py", line 1137, in __init__
    adapter_cls = select_data_adapter(x, y)
  File "/home/miran045/reine097/projects/ResNet34/venv/lib/python3.6/site-packages/keras/engine/data_adapter.py", line 979, in select_data_adapter
    _type_name(x), _type_name(y)))
ValueError: Failed to find data adapter that can handle input: (<class 'list'> containing values of types {"<class 'numpy.ndarray'>"}), (<class 'list'> containing values of types {"<class 'int'>"})
python-BaseException

完整代码如下：

from functools import partial

import numpy as np
import tensorflow as tf
from tensorflow import keras

DefaultConv3D = partial(keras.layers.Conv3D, kernel_size=3, strides=1,
                        padding="SAME", use_bias=False)


class ResidualUnit(keras.layers.Layer):
    def __init__(self, filters, strides=1, activation="relu", **kwargs):
        super().__init__(**kwargs)
        self.activation = keras.activations.get(activation)
        self.main_layers = [
            DefaultConv3D(filters, strides=strides),
            keras.layers.BatchNormalization(),
            self.activation,
            DefaultConv3D(filters),
            keras.layers.BatchNormalization()]
        self.skip_layers = []
        if strides > 1:
            self.skip_layers = [
                DefaultConv3D(filters, kernel_size=1, strides=strides),
                keras.layers.BatchNormalization()]

    def call(self, inputs):
        Z = inputs
        for layer in self.main_layers:
            Z = layer(Z)
        skip_Z = inputs
        for layer in self.skip_layers:
            skip_Z = layer(skip_Z)
        return self.activation(Z + skip_Z)


def get_model():
    model = keras.models.Sequential()
    model.add(DefaultConv3D(64, kernel_size=7, strides=2,
                            input_shape=[1, 182, 218, 182]))
    model.add(keras.layers.BatchNormalization())
    model.add(keras.layers.Activation("relu"))
    model.add(keras.layers.MaxPool3D(pool_size=3, strides=2, padding="SAME"))
    prev_filters = 64
    for filters in [64] * 3 + [128] * 4 + [256] * 6 + [512] * 3:
        strides = 1 if filters == prev_filters else 2
        model.add(ResidualUnit(filters, strides=strides))
        prev_filters = filters
    model.add(keras.layers.GlobalAvgPool3D())
    model.add(keras.layers.Flatten())
    model.add(keras.layers.Dense(1))

    return model


def run():
    model = get_model()
    model.compile(loss="mean_squared_error", optimizer="adam", metrics=[tf.keras.metrics.MeanSquaredError()])

    x1 = np.random.rand(182, 218, 182)
    x2 = np.random.rand(182, 218, 182)
    x3 = np.random.rand(182, 218, 182)
    x4 = np.random.rand(182, 218, 182)
    x5 = np.random.rand(182, 218, 182)
    x6 = np.random.rand(182, 218, 182)
    X_train = [x1, x2, x3]
    y_train = [2]
    X_valid = [x4]
    y_valid = [2]
    X_test = [x5]
    y_test = [3]
    history = model.fit(X_train, y_train, epochs=10, validation_data=(X_valid, y_valid))
    print(history)
    score = model.evaluate(X_test, y_test)
    print(score)
    X_new = x6
    y_pred = model.predict(X_new)
    print(y_pred)


if __name__ == '__main__':
    run()

请问哪里出错了？该如何转换训练和验证数据以解决这个问题？

错误原因 #

1. 输入格式不兼容：Keras的fit方法要求输入为批量格式的numpy数组，而非普通列表。你当前的X_train/X_valid是numpy数组的列表，y_train是整数列表，不符合模型的数据适配要求。
2. 维度不匹配：模型定义的输入形状是[1, 182, 218, 182]，但生成的单个样本是(182,218,182)，缺少通道维度和批量维度，无法被模型识别。
3. 标签数量不匹配：X_train有3个样本，但y_train只有1个标签，后续会触发样本与标签数量不一致的错误。

解决步骤 #

1. 调整样本维度

给单个样本添加通道维度，使其匹配模型输入的形状要求：将(182,218,182)扩展为(1,182,218,182)。

2. 转换为批量numpy数组

将多个样本堆叠成包含批量维度的numpy数组，标签也转为对应形状的numpy数组。

修改后的run函数示例 #

def run():
    model = get_model()
    model.compile(loss="mean_squared_error", optimizer="adam", metrics=[tf.keras.metrics.MeanSquaredError()])

    # 生成带通道维度的单个样本
    x1 = np.random.rand(1, 182, 218, 182)
    x2 = np.random.rand(1, 182, 218, 182)
    x3 = np.random.rand(1, 182, 218, 182)
    x4 = np.random.rand(1, 182, 218, 182)
    x5 = np.random.rand(1, 182, 218, 182)
    x6 = np.random.rand(1, 182, 218, 182)
    
    # 堆叠为批量数组
    X_train = np.concatenate([x1, x2, x3], axis=0)
    y_train = np.array([2, 2, 2]).reshape(-1, 1)  # 每个样本对应一个标签，转为二维数组
    X_valid = np.concatenate([x4], axis=0)
    y_valid = np.array([2]).reshape(-1, 1)
    X_test = np.concatenate([x5], axis=0)
    y_test = np.array([3]).reshape(-1, 1)
    
    history = model.fit(X_train, y_train, epochs=10, validation_data=(X_valid, y_valid))
    print(history)
    score = model.evaluate(X_test, y_test)
    print(score)
    
    # 预测时保持输入维度正确
    y_pred = model.predict(x6)
    print(y_pred)

额外说明 #

如果习惯使用通道最后（Keras默认格式），可以修改模型输入形状为[182,218,182,1]，同时将单个样本扩展为(182,218,182,1)，这样更符合常规数据格式。

内容的提问来源于stack exchange，提问作者Paul Reiners