基于CNN的小数据集图像分类模型准确率偏低的优化咨询

问题背景 #

尝试构建6分类图像CNN分类器，数据集规模较小：训练集310张、验证集78张、测试集100张；训练与验证集按类别命名子文件夹，测试集子文件夹为A-F。调整epochs参数后，模型测试准确率最高仅20%，以下为实现代码及优化建议：

原始代码（含修复点） #

!pip install keras
!pip install tensorflow

from google.colab import drive
drive.mount('/content/drive')

#importing Libraries
import numpy as np
import pandas as pd
import os
import random

#import Library for data Visualization
import matplotlib.image as mpimg
import matplotlib.pyplot as plt
import seaborn as sns
import cv2

#transporting Tensorflow for modeal creation and its dependencies
import tensorflow

#Generate batches of tensor image data with real-time data augmentation
from keras.preprocessing.image import ImageDataGenerator

#for enabling inline plotting
# %matplotlib inline

TrainingImagePath = '/content/drive/MyDrive/Training'
TestingImagePath = '/content/drive/MyDrive/Test'
ValidationImagePath = '/content/drive/MyDrive/Validation'

# Defining the pre-processing transformation on raw images of training data
train_datagen = ImageDataGenerator(
    rescale=1./255,
    shear_range=0.1,
    zoom_range=0.1,
    horizontal_flip=True
)

# Defining pre-processing transformations on raw images of testing data
test_datagen = ImageDataGenerator(rescale=1./255)

# Generating the Training Data
training_set = train_datagen.flow_from_directory(
    TrainingImagePath,
    target_size=(128, 128),
    batch_size=32,
    class_mode='categorical'
)

# Generating the Testing Data
test_set = test_datagen.flow_from_directory(
    TestingImagePath,
    target_size=(128, 128),
    batch_size=32,
    class_mode='categorical'
)

# Generating the Validation Data
valid_set = test_datagen.flow_from_directory(
    ValidationImagePath,
    target_size=(128, 128),
    batch_size=32,
    class_mode='categorical'
)


def showImages(class_name):
    folder_path = os.path.join(TrainingImagePath, class_name)
    images_list = os.listdir(folder_path)

    random_index = random.choice(images_list)
    image_path = os.path.join(folder_path, random_index)

    plt.imshow(mpimg.imread(image_path))
    plt.title(class_name)
    plt.axis(False)

plt.figure(figsize=(20, 20))
for labels, number in training_set.class_indices.items():
    plt.subplot(6, 6, number+1)
    showImages(labels)

TrainClasses=training_set.class_indices

ResultMap={}
for ballvalue, ballName in zip(TrainClasses.values(), TrainClasses.keys()):
  ResultMap[ballvalue]=ballName

import pickle
with open(R"/content/drive/MyDrive/Results.pk1", "wb") as f:
  pickle.dump(ResultMap, f, pickle.HIGHEST_PROTOCOL)

print("Mapping of Face and its ID", ResultMap)

OutputNeurons=len(ResultMap)
print('\n The number of output neurons:', OutputNeurons)

from keras.models import Sequential
from keras.layers import Convolution2D
from keras.layers import MaxPool2D
from keras.layers import Flatten
from keras.layers import Dense

classifier=Sequential()

classifier.add(Convolution2D(32, kernel_size=(3,3), strides=(1,1), input_shape=(128,128,3), activation="relu"))

classifier.add(MaxPool2D(pool_size=(2,2)))

classifier.add(Convolution2D(64, kernel_size=(3,3), strides=(1,1),activation="relu"))

classifier.add(MaxPool2D(pool_size=(2,2)))

classifier.add(Flatten())

classifier.add(Dense(256, activation='relu'))

classifier.add(Dense(OutputNeurons, activation="softmax"))

classifier.compile(loss="categorical_crossentropy", optimizer="rmsprop", metrics=["accuracy"])

classifier.summary()

import time

# Measuring the time taken by the model to train
StartTime = time.time()

# Starting the model training
# 修复：替换已弃用的fit_generator为fit
model_history = classifier.fit(
    training_set,
    steps_per_epoch=len(training_set),
    epochs=20,
    validation_data=valid_set,
    validation_steps=len(valid_set),
    verbose=1
)

EndTime = time.time()
# 修复：定义total_time_minutes变量
total_time_minutes = (EndTime - StartTime)/60
print('############# Total Time Taken: {:.2f} Minutes ###############'.format(total_time_minutes))

accuracy=model_history.history['accuracy']
val_accuracy=model_history.history['val_accuracy']

loss=model_history.history['loss']
val_loss=model_history.history['val_loss']

plt.figure(figsize=(15,10))

plt.subplot(2,2,1)
plt.plot(accuracy,label="Training accuracy")
plt.plot(val_accuracy, label="Validation accuracy")
plt.legend()
plt.title ("Training vs validation accuracy")

plt.subplot(2,2,2)
plt.plot(loss, label="Training loss")
plt.plot(val_loss, label="Validation loss")
plt.legend()
plt.title("Training vs validation loss")

plt.show()

classifier.save("/content/drive/MyDrive/Classifier.h5")
# 修复：Keras模型建议保存为.h5格式，而非pk1

核心优化方案 #

1. 数据集层面优化 #

• 强化数据扩充：当前扩充手段单一，增加更多变换提升样本多样性：

  train_datagen = ImageDataGenerator(
      rescale=1./255,
      shear_range=0.2,
      zoom_range=0.2,
      horizontal_flip=True,
      vertical_flip=True,
      rotation_range=20,
      width_shift_range=0.1,
      height_shift_range=0.1,
      brightness_range=[0.8, 1.2]
  )
  

• 检查类别平衡：统计每个类别的样本数量，若存在失衡，可采用过采样少数类、欠采样多数类，或在训练时指定class_weight参数：

  # 示例：假设类别0样本最少，设置权重
  class_weight = {0: 3.0, 1:1.0, 2:1.0, 3:1.0, 4:1.0, 5:1.0}
  model_history = classifier.fit(..., class_weight=class_weight)
  

• 验证测试集映射：确认测试集A-F的类别与训练集完全对应，避免标签不匹配导致准确率计算错误。

2. 模型结构优化 #

• 迁移学习（优先级最高）：小数据集下直接训练自定义CNN效果差，使用预训练模型提取特征：

  from tensorflow.keras.applications import MobileNetV2
  from tensorflow.keras.layers import GlobalAveragePooling2D, Dropout
  
  base_model = MobileNetV2(weights='imagenet', include_top=False, input_shape=(128,128,3))
  base_model.trainable = False  # 冻结预训练层
  
  classifier = Sequential([
      base_model,
      GlobalAveragePooling2D(),
      Dense(128, activation='relu'),
      Dropout(0.5),
      Dense(OutputNeurons, activation="softmax")
  ])
  

• 抑制过拟合：在全连接层后添加Dropout层，减少神经元数量：

  classifier.add(Flatten())
  classifier.add(Dense(128, activation='relu'))
  classifier.add(Dropout(0.5))
  classifier.add(Dense(OutputNeurons, activation="softmax"))
  

• 增强特征提取：新增1-2层卷积层提升特征捕捉能力：

  classifier.add(Convolution2D(32, kernel_size=(3,3), strides=(1,1), input_shape=(128,128,3), activation="relu"))
  classifier.add(MaxPool2D(pool_size=(2,2)))
  classifier.add(Convolution2D(64, kernel_size=(3,3), strides=(1,1),activation="relu"))
  classifier.add(MaxPool2D(pool_size=(2,2)))
  classifier.add(Convolution2D(128, kernel_size=(3,3), strides=(1,1),activation="relu"))
  classifier.add(MaxPool2D(pool_size=(2,2)))
  

3. 训练策略优化 #

• 优化器与学习率调整：更换为Adam优化器并设置较小学习率：

  from tensorflow.keras.optimizers import Adam
  classifier.compile(loss="categorical_crossentropy", optimizer=Adam(learning_rate=1e-4), metrics=["accuracy"])
  

• 早停机制：避免过拟合，当验证集准确率不再提升时停止训练：

  from tensorflow.keras.callbacks import EarlyStopping
  early_stop = EarlyStopping(monitor='val_accuracy', patience=5, restore_best_weights=True)
  model_history = classifier.fit(..., callbacks=[early_stop], epochs=50)
  

4. 评估与调试 #

• 混淆矩阵分析：明确模型在哪些类别上表现最差，针对性优化：

  from sklearn.metrics import confusion_matrix, classification_report
  
  y_pred = classifier.predict(test_set)
  y_pred_classes = np.argmax(y_pred, axis=1)
  y_true = test_set.classes
  
  cm = confusion_matrix(y_true, y_pred_classes)
  plt.figure(figsize=(8,6))
  sns.heatmap(cm, annot=True, fmt='d', xticklabels=ResultMap.values(), yticklabels=ResultMap.values())
  plt.xlabel('Predicted')
  plt.ylabel('True')
  plt.show()
  
  print(classification_report(y_true, y_pred_classes, target_names=ResultMap.values()))
  

内容的提问来源于stack exchange，提问作者bscalingi