基于VGG-16特征提取的Keras二分类任务样本数不匹配报错

嘿，我一眼就发现问题所在了——你在调用predict_generator时传入的steps参数完全错了，这才导致特征样本数和标签样本数不匹配，触发了那个ValueError。

问题根源 #

predict_generator的steps参数代表的是要执行的批次数，而不是总样本数。你的batch_size设置为10，训练集总共有1000个样本，所以需要的steps应该是1000 // 10 = 100，但你直接传了1000，这就相当于让模型跑1000个批次，每个批次10个样本，最终生成了1000*10=10000个特征样本，和你准备的1000个标签完全对不上，不报错才怪呢！

修复后的完整代码 #

我已经把训练集和测试集的predict_generator参数都修正好了，直接用下面的代码替换你的原代码就行：

########################load libraries#########################################
import numpy as np
import time
from keras.layers import Dense, Dropout, Flatten
from keras.models import Model
from keras import applications
from keras.optimizers import SGD
from keras.preprocessing.image import ImageDataGenerator
#########################image characteristics#################################
img_rows=100 #dimensions of image, to be varied suiting the input requirements of the pre-trained model
img_cols=100
channel = 3 #RGB
num_classes = 2
batch_size = 10
nb_epoch = 10
###############################################################################
''' This code uses VGG-16 as a feature extractor'''
feature_model = applications.VGG16(weights='imagenet', include_top=False, input_shape=(img_rows, img_cols, 3))
#get the model summary
feature_model.summary()
#extract feature from the intermediate layer
feature_model = Model(input=feature_model.input, output=feature_model.get_layer('block5_conv2').output)
#get the model summary
feature_model.summary()
#declaring image data generators
train_datagen = ImageDataGenerator()
val_datagen = ImageDataGenerator()

# 修复训练集的predict_generator steps参数
generator = train_datagen.flow_from_directory(
    'f1_100/train',
    target_size=(img_rows, img_cols),
    batch_size=batch_size,
    class_mode=None,
    shuffle=False)
# steps = 总训练样本数 // batch_size
train_data = feature_model.predict_generator(generator, steps=1000 // batch_size)
train_labels = np.array([[1, 0]] * 500 + [[0, 1]] * 500)

# 修复测试集的predict_generator steps参数
generator = val_datagen.flow_from_directory(
    'f1_100/test',
    target_size=(img_rows, img_cols),
    batch_size=batch_size,
    class_mode=None,
    shuffle=False)
# steps = 总测试样本数 // batch_size
validation_data = feature_model.predict_generator(generator, steps=200 // batch_size)
validation_labels = np.array([[1,0]] * 100 + [[0,1]] * 100)
###############################################################################
#addding the top layers and training them on the extracted features
from keras.models import Sequential
model = Sequential()
model.add(Flatten(input_shape=train_data.shape[1:]))
model.add(Dense(1024, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(2, activation='softmax'))
sgd = SGD(lr=0.0001, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])
print('-'*30)
print('Start Training the top layers on the extracted features...')
print('-'*30)
#measure the time and train the model
t=time.time()
hist = model.fit(train_data, train_labels, nb_epoch=nb_epoch, batch_size=batch_size, validation_data=(validation_data, validation_labels), verbose=2)
#print the history of the trained model
print(hist.history)
print('Training time: %s' % (time.time()-t))
###############################################################################

额外小提示 #

1. 其实在较新版本的Keras中，你可以不用手动指定steps，它会自动根据生成器里的总样本数和batch_size计算，但显式计算出来会更稳妥，避免版本兼容问题。
2. 记得确认一下f1_100/train和f1_100/test目录下的样本数确实是1000和200，要是目录结构错了，样本数不对的话也可能出问题哦。

内容的提问来源于stack exchange，提问作者shiva