使用Keras 3 API运行CNN的fit方法时遇未识别数据类型错误如何解决
图像分类CNN训练报错:Unrecognized data type
我正在跟随在线课程,使用Python 3.11、Keras 3.6和TensorFlow 2.18构建图像分类CNN,代码如下:
# Convolutional Nueral Network import tensorflow as tf import keras as kr from tf_keras.preprocessing.image import ImageDataGenerator import numpy as np from tf_keras.preprocessing import image print(tf.__version__) print(kr.__version__) # Part 1 - Data Preprocessing # Preprocessing the Training Set # Below, an instance of the class of ImageDataGenerator that causes transformations train_datagen = ImageDataGenerator( rescale=1./255, shear_range=0.2, zoom_range=0.2, horizontal_flip=True) training_set = train_datagen.flow_from_directory( "dataset/training_set", target_size=(64,64), batch_size=32, class_mode='binary') # Preprocessing the Test Set test_datagen = ImageDataGenerator(rescale=1./255) test_set = test_datagen.flow_from_directory( "dataset/test_set", target_size=(64,64), batch_size=32, class_mode='binary') # Part 2 - Building the CNN # Initializing the CNN cnn = kr.Sequential() # Step 1 - Convolution cnn.add(kr.layers.Conv2D(filters=32, kernel_size=3, activation='relu', input_shape=[64,64,3])) # Step 2 - Pooling cnn.add(kr.layers.MaxPool2D(pool_size=2, strides=2, padding='valid')) # Step 3 - Adding a second convolutional layer cnn.add(kr.layers.Conv2D(filters=32, kernel_size=3, activation='relu')) # Step 4 - Adding a second pooling layer cnn.add(kr.layers.MaxPool2D(pool_size=2, strides=2, padding='valid')) # Step 5 Flattening cnn.add(kr.layers.Flatten()) # Step 6 Full Connection cnn.add(kr.layers.Dense(units=128, activation='relu')) # Step 7 Output Layer cnn.add(kr.layers.Dense(units=1, activation='sigmoid')) #Sigmoid b/c classification is binary # Part 3 - Training the CNN # Step 1 Compiling the CNN cnn.compile(optimizer='adam', loss = 'binary_crossentropy', metrics= ['accuracy']) # Step 2 Training the CNN on the Training Set and Evaluating it on the Test Set cnn.fit(x = training_set, validation_data = test_set, epochs=25)
运行至Part 3的cnn.fit步骤时,出现如下错误:
raise ValueError(f"Unrecognized data type: x={x} (of type {type(x)})") ValueError: Unrecognized data type: x=<tf_keras.src.preprocessing.image.DirectoryIterator object at 0x141d69210> (of type <class 'tf_keras.src.preprocessing.image.DirectoryIterator'>)
查阅Keras 3文档得知fit方法仅支持特定数据类型,尝试将数据集转为numpy数组传入fit方法,却耗时极久且占用大量内存,以下是解决方案:
核心原因
Keras 3不再兼容旧的tf_keras.preprocessing.image.DirectoryIterator类型,需要改用TensorFlow原生的数据集加载方式或者适配Keras 3的数据格式。
解决方案1:改用TensorFlow的tf.keras.utils.image_dataset_from_directory(推荐)
这是Keras 3官方推荐的数据集加载方式,直接返回兼容fit方法的tf.data.Dataset对象,内存效率更高,且支持将数据增强整合到模型中。替换原数据预处理部分的代码:
# 替换原Data Preprocessing部分 import tensorflow as tf import keras as kr # 训练集加载(自动处理目录结构) training_set = tf.keras.utils.image_dataset_from_directory( "dataset/training_set", image_size=(64, 64), # 统一图像尺寸 batch_size=32, label_mode='binary', # 二分类任务 shuffle=True # 训练时打乱数据 ) # 定义数据增强层(Keras 3推荐方式) data_augmentation = kr.Sequential([ kr.layers.RandomFlip("horizontal"), # 随机水平翻转 kr.layers.RandomZoom(0.2), # 随机缩放 kr.layers.RandomShear(0.2) # 随机剪切 ]) # 将数据增强与归一化整合到训练集 def preprocess_train(x, y): x = data_augmentation(x, training=True) # 训练模式下应用增强 x = kr.layers.Rescaling(1./255)(x) # 归一化到0-1 return x, y training_set = training_set.map(preprocess_train, num_parallel_calls=tf.data.AUTOTUNE) # 测试集加载(无需打乱) test_set = tf.keras.utils.image_dataset_from_directory( "dataset/test_set", image_size=(64, 64), batch_size=32, label_mode='binary', shuffle=False ) # 测试集仅做归一化 def preprocess_test(x, y): x = kr.layers.Rescaling(1./255)(x) return x, y test_set = test_set.map(preprocess_test, num_parallel_calls=tf.data.AUTOTUNE) # 优化数据集性能(预取数据,提升训练速度) training_set = training_set.prefetch(tf.data.AUTOTUNE) test_set = test_set.prefetch(tf.data.AUTOTUNE)
修改后,原CNN模型代码无需变动,直接调用cnn.fit(training_set, validation_data=test_set, epochs=25)即可正常训练。
解决方案2:将DirectoryIterator转换为tf.data.Dataset(兼容旧代码)
如果需要保留原ImageDataGenerator的配置,可以将迭代器转换为tf.data.Dataset:
# 定义转换函数 def iterator_to_dataset(iterator): def gen(): for x, y in iterator: yield x, y # 指定输出张量的形状和类型 return tf.data.Dataset.from_generator( gen, output_signature=( tf.TensorSpec(shape=(None, 64, 64, 3), dtype=tf.float32), tf.TensorSpec(shape=(None,), dtype=tf.float32) ) ) # 转换训练集和测试集 training_set = iterator_to_dataset(training_set) test_set = iterator_to_dataset(test_set) # 优化性能 training_set = training_set.prefetch(tf.data.AUTOTUNE) test_set = test_set.prefetch(tf.data.AUTOTUNE)
转换完成后即可正常调用cnn.fit,但这种方式不如方案1简洁高效,仅作为临时兼容方案使用。
内容的提问来源于stack exchange,提问作者katmatzidis
相关产品推荐
相关产品推荐

