image_dataset_from_directory无法读取PNG/JPEG，Colab正常VS Code异常求助

问题详情 #

代码在Colab中运行正常，但本地VS Code执行时触发格式错误，核心报错信息：

tensorflow.python.framework.errors_impl.InvalidArgumentError: Unknown image file format. One of JPEG, PNG, GIF, BMP required.

用户代码：

image_dir = "ai/data/toonder_img"
batch_size = 32
image_size = (64, 64)

train_ds = image_dataset_from_directory(
    directory=image_dir,
    labels="inferred",
    color_mode='grayscale',
    class_names=['class'],
    image_size=image_size,
    batch_size=batch_size,
    seed=1337
    )

class_names = train_ds.class_names
take = train_ds.take(1)

print(train_ds.element_spec)
print("class_names : ", class_names)
print("data_mode   : ", take)

for images in train_ds.take(10):
  print('images.shape: ', images.shape)

完整错误输出：

Found 293 files belonging to 1 classes.
(TensorSpec(shape=(None, 64, 64, 1), dtype=tf.float32, name=None), TensorSpec(shape=(None,), dtype=tf.int32, name=None))
class_names :  ['class']
data_mode   :  <_TakeDataset element_spec=(TensorSpec(shape=(None, 64, 64, 1), dtype=tf.float32, name=None), TensorSpec(shape=(None,), dtype=tf.int32, name=None))>
Traceback (most recent call last):
  File "/Users/kite/Desktop/2023_2_Toonder2.0/ai/toonder_module/toonder_img_ai.py", line 42, in <module>
    for images in train_ds.take(10):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 809, in __next__
    return self._next_internal()
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 772, in _next_internal
    ret = gen_dataset_ops.iterator_get_next(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 3028, in iterator_get_next
    _ops.raise_from_not_ok_status(e, name)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/tensorflow/python/framework/ops.py", line 5888, in raise_from_not_ok_status
    raise core._status_to_exception(e) from None  # pylint: disable=protected-access
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__IteratorGetNext_output_types_2_device_/job:localhost/replica:0/task:0/device:CPU:0}} Unknown image file format. One of JPEG, PNG, GIF, BMP required.
         [[{{node decode_image/DecodeImage}}]] [Op:IteratorGetNext] name:

解决方法 #

1. 清理目录中的隐藏/非图片文件 #

本地系统会自动生成隐藏文件（比如Mac的.DS_Store、Windows的Thumbs.db），这些文件会被image_dataset_from_directory当作图片读取，触发格式错误。可以手动删除，也用脚本批量清理：

import os
from pathlib import Path

# 遍历目标目录及其子目录，删除所有隐藏文件
for file in Path(image_dir).rglob(".*"):
    if file.is_file():
        os.remove(file)

2. 检查图片文件完整性 #

部分图片可能存在损坏（下载中断、存储错误），Colab环境可能有容错逻辑，但本地TensorFlow解码更严格。用以下脚本批量验证并清理损坏图片：

from PIL import Image
import os

# 注意替换为实际的图片子目录（因为image_dataset_from_directory要求类目录结构）
image_subdir = os.path.join(image_dir, "class")
for filename in os.listdir(image_subdir):
    if filename.lower().endswith(('.png', '.jpg', '.jpeg')):
        file_path = os.path.join(image_subdir, filename)
        try:
            with Image.open(file_path) as img:
                img.verify()  # 验证文件完整性
        except (IOError, SyntaxError) as e:
            print(f"发现损坏图片: {filename}")
            # 可选：直接删除损坏文件
            # os.remove(file_path)

3. 对齐TensorFlow版本 #

Colab的TensorFlow版本和本地可能不一致，导致解码逻辑差异。先查看Colab的版本：

!pip show tensorflow

然后在本地安装对应版本：

pip install tensorflow==x.x.x  # 替换为Colab显示的版本号

4. 自定义图片加载逻辑（兜底方案） #

如果上述方法无效，可以绕过image_dataset_from_directory的自动加载，自定义加载函数显式处理图片格式：

import tensorflow as tf
from pathlib import Path

image_dir = "ai/data/toonder_img"
batch_size = 32
image_size = (64, 64)

def load_image(file_path):
    # 读取文件
    img = tf.io.read_file(file_path)
    # 根据后缀显式解码
    if tf.strings.regex_full_match(file_path, ".*\\.png"):
        img = tf.image.decode_png(img, channels=1)
    else:
        img = tf.image.decode_jpeg(img, channels=1)
    # 调整尺寸并归一化
    img = tf.image.resize(img, image_size)
    img = tf.cast(img, tf.float32) / 255.0
    return img

# 获取所有图片路径
image_paths = list(Path(image_dir).glob("class/*.png")) + list(Path(image_dir).glob("class/*.jpg")) + list(Path(image_dir).glob("class/*.jpeg"))
# 单类别标签全为0
labels = [0] * len(image_paths)

# 构建数据集
train_ds = tf.data.Dataset.from_tensor_slices(([str(p) for p in image_paths], labels))
train_ds = train_ds.map(lambda x, y: (load_image(x), y), num_parallel_calls=tf.data.AUTOTUNE)
train_ds = train_ds.shuffle(1000).batch(batch_size)

# 验证数据集
for images, labels in train_ds.take(1):
    print(images.shape, labels.shape)

内容的提问来源于stack exchange，提问作者kitewatermelon