使用tensorflow.keras.ImageDataGenerator图像增强遇0有效图像文件名问题

我需要为手动标注的RIWA数据集生成增强版本，已完成数据集整理和采样步骤，但在运行图像增强代码时，长时间等待后仍提示“Found 0 validated image filenames”。已尝试修正Keras导入路径、统一图像尺寸，但问题依旧，怀疑是否和CPU版TensorFlow有关。

已执行的代码步骤 #

1. 数据集整理代码 #

source_dir = r'./river-water-segmentation-dataset/riwa_v2'
subdir = os.listdir(source_dir)

filepaths = []
labels = []

for i in subdir:
    classpath = os.path.join(source_dir, i)

    if os.path.isdir(classpath):
        file_list = os.listdir(classpath)
        for f in file_list:
            file_path = os.path.join(classpath, f)
            filepaths.append(file_path)
            labels.append(i)
paths = pd.Series(filepaths, name='paths')
labels = pd.Series(labels, name='labels')

df = pd.concat([paths, labels], axis=1)

print(df.head())
print("========================")
print(df['labels'].value_counts())
print("=========================")
print('Total data: ', len(df))

2. 数据集采样代码 #

sample_list = []
max_size = 1500# TODO: change this value

grouping = df.groupby('labels')

for label in df['labels'].unique():
    group = grouping.get_group(label)
    group_size = len(group)

    if group_size > max_size:
        samples = group.sample(max_size, replace=False, weights=None, axis=0).reset_index(drop=True)
    else:
        samples = group.sample(frac=1.0, replace=False, axis=0).reset_index(drop=True)
    sample_list.append(samples)

df = pd.concat(sample_list, axis=0).reset_index(drop=True)
print(df['labels'].value_counts())
print('Total data: ', len(df))

3. 图像增强代码 #

import os
import shutil
from tensorflow.keras.preprocessing.image import ImageDataGenerator

working_dir = r'./river-water-segmentation-dataset/riwa_v2/cropped'

aug_dir = os.path.join(working_dir, 'aug')
if os.path.isdir(aug_dir):
    shutil.rmtree(aug_dir)
os.mkdir(aug_dir)
for label in df['labels'].unique():
    dir_path=os.path.join(aug_dir, label)
    os.mkdir(dir_path)
print(os.listdir(aug_dir))

target = 700 # set the target count for each class in df
gen = ImageDataGenerator(
    rotation_range = 90,
    horizontal_flip = True,
    vertical_flip = True,
)

grouping = df.groupby('labels') # group by class


for label in df['labels'].unique(): # for every class
    group = grouping.get_group(label) # a dataframe holding only rows with the specificied label
    sample_count = len(group) # determine how many samples there are in this class
    # if group.empty:
    #     print(f"No images found for label '{label}'. Skipping augmentation.")
    #     continue
    if sample_count < target: # if the class has less than target number of images
        aug_img_count = 0
        delta = target - sample_count # number of augmented images to create
        target_dir = os.path.join(aug_dir, label) # define where to write the images

        aug_gen = gen.flow_from_dataframe(
            group,
            x_col = 'paths',
            y_col = None,
            target_size = (1420, 1080), # change this target size based on transfer learning model
            class_mode = None,
            batch_size = 1,
            shuffle = False,
            save_to_dir = target_dir,
            save_prefix = 'aug-',
            save_format='jpg'
        )
        images = next(aug_gen)  # Try fetching a batch
        print(f"Generated {len(images)} images.")

        while aug_img_count < delta:
            images = next(aug_gen)
            aug_img_count += len(images)

解决方案 #

1. 验证数据路径有效性 #

• 先检查df中存储的图像路径是否真实存在：

df['path_valid'] = df['paths'].apply(os.path.exists)
print(f"无效路径数量：{len(df[df['path_valid'] == False])}")
print(df[df['path_valid'] == False]['paths'])

• 确保working_dir和aug_dir的路径正确，且当前用户拥有该目录的读写权限。

2. 检查图像格式兼容性 #

• 确认数据集图像格式是否与save_format='jpg'兼容，先统计图像扩展名：

df['ext'] = df['paths'].apply(lambda x: os.path.splitext(x)[1].lower())
print(df['ext'].value_counts())

• 若存在PNG等其他格式，可修改ImageDataGenerator的color_mode参数（如color_mode='rgba'适配透明图像），或统一转换图像格式后再执行增强。

3. 修正flow_from_dataframe参数 #

• 确认target_size的顺序为**(高度, 宽度)**，需与裁剪后图像的实际尺寸完全匹配。
• 若路径包含中文或特殊字符，会导致TensorFlow无法读取，建议将数据集迁移到纯英文路径下。
• 添加validate_filenames=True参数显式验证文件名：

aug_gen = gen.flow_from_dataframe(
    group,
    x_col = 'paths',
    y_col = None,
    target_size = (1420, 1080),
    class_mode = None,
    batch_size = 1,
    shuffle = False,
    save_to_dir = target_dir,
    save_prefix = 'aug-',
    save_format='jpg',
    validate_filenames=True
)

4. 排除CPU版TensorFlow的影响 #

CPU版TensorFlow不会直接导致“0有效文件名”问题，但会大幅减慢处理速度。建议先取10条样本数据测试代码逻辑，确认路径和参数无误后再处理全量数据。

5. 调试生成器状态 #

在创建aug_gen后，直接打印生成器的有效样本数，快速定位问题：

print(f"标签 {label} 有效样本数：{aug_gen.samples}")

若输出为0，说明路径验证完全失败，需回到数据集整理步骤检查路径拼接逻辑。

内容的提问来源于stack exchange，提问作者RedSean