You need to enable JavaScript to run this app. 如何将自定义图像数据集及对应CSV标签载入PyTorch或Keras
优惠活动
大模型
产品
解决方案
定价
更多

如何将自定义图像数据集及对应CSV标签载入PyTorch或Keras

自定义数据集加载实现方案

前提说明:假设你的标签CSV结构为文件名,标签两列,训练图像文件夹、训练标签CSV、测试图像文件夹、测试标签CSV路径已提前配置。

方案1:Keras/TensorFlow 实现

直接封装load_data函数即可实现你想要的调用效果,代码示例如下:

import os
import cv2
import pandas as pd
import numpy as np

def load_custom_data(train_img_dir, train_label_path, test_img_dir, test_label_path, img_size=(28,28), is_grayscale=True):
    # 加载训练集
    train_label_df = pd.read_csv(train_label_path)
    train_images = []
    train_labels = []
    for _, row in train_label_df.iterrows():
        img_path = os.path.join(train_img_dir, row['filename'])
        # 读入图片
        if is_grayscale:
            img = cv2.imread(img_path, cv2.IMREAD_GRAYSCALE)
        else:
            img = cv2.cvtColor(cv2.imread(img_path), cv2.COLOR_BGR2RGB)
        # 统一尺寸
        img = cv2.resize(img, img_size)
        train_images.append(img)
        train_labels.append(row['label'])
    # 转成numpy数组,和MNIST返回格式对齐
    train_images = np.array(train_images)
    train_labels = np.array(train_labels)
    
    # 加载测试集逻辑和训练集完全一致
    test_label_df = pd.read_csv(test_label_path)
    test_images = []
    test_labels = []
    for _, row in test_label_df.iterrows():
        img_path = os.path.join(test_img_dir, row['filename'])
        if is_grayscale:
            img = cv2.imread(img_path, cv2.IMREAD_GRAYSCALE)
        else:
            img = cv2.cvtColor(cv2.imread(img_path), cv2.COLOR_BGR2RGB)
        img = cv2.resize(img, img_size)
        test_images.append(img)
        test_labels.append(row['label'])
    test_images = np.array(test_images)
    test_labels = np.array(test_labels)
    
    return (train_images, train_labels), (test_images, test_labels)

调用方式和你想要的MNIST加载逻辑完全一致:

# 替换成你自己的路径
TRAIN_IMG_DIR = r"你的训练图像文件夹路径"
TRAIN_LABEL_PATH = r"你的训练标签CSV路径"
TEST_IMG_DIR = r"你的测试图像文件夹路径"
TEST_LABEL_PATH = r"你的测试标签CSV路径"

(train_images, train_labels), (test_images, test_labels) = load_custom_data(
    TRAIN_IMG_DIR, TRAIN_LABEL_PATH, TEST_IMG_DIR, TEST_LABEL_PATH
)

方案2:PyTorch 实现

你可以继承PyTorch的Dataset类封装自定义数据集,再封装加载函数即可:

import os
import cv2
import pandas as pd
from torch.utils.data import Dataset, DataLoader

class CustomImageDataset(Dataset):
    def __init__(self, img_dir, label_path, img_size=(28,28), is_grayscale=True, transform=None):
        self.img_dir = img_dir
        self.label_df = pd.read_csv(label_path)
        self.img_size = img_size
        self.is_grayscale = is_grayscale
        self.transform = transform

    def __len__(self):
        return len(self.label_df)

    def __getitem__(self, idx):
        img_path = os.path.join(self.img_dir, self.label_df.iloc[idx, 0])
        if self.is_grayscale:
            image = cv2.imread(img_path, cv2.IMREAD_GRAYSCALE)
        else:
            image = cv2.cvtColor(cv2.imread(img_path), cv2.COLOR_BGR2RGB)
        image = cv2.resize(image, self.img_size)
        label = self.label_df.iloc[idx, 1]
        if self.transform:
            image = self.transform(image)
        return image, label

def load_custom_data(train_img_dir, train_label_path, test_img_dir, test_label_path, batch_size=32, transform=None):
    train_dataset = CustomImageDataset(train_img_dir, train_label_path, transform=transform)
    test_dataset = CustomImageDataset(test_img_dir, test_label_path, transform=transform)
    train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
    test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)
    return train_loader, test_loader

调用方式:

# 替换成你自己的路径
TRAIN_IMG_DIR = r"你的训练图像文件夹路径"
TRAIN_LABEL_PATH = r"你的训练标签CSV路径"
TEST_IMG_DIR = r"你的测试图像文件夹路径"
TEST_LABEL_PATH = r"你的测试标签CSV路径"

train_loader, test_loader = load_custom_data(TRAIN_IMG_DIR, TRAIN_LABEL_PATH, TEST_IMG_DIR, TEST_LABEL_PATH)

如果不需要DataLoader,也可以参考Keras的实现逻辑直接返回张量格式的训练、测试数据和标签,和MNIST的调用格式完全对齐。
另外你之前写的代码存在缩进错误,img_array = cv2.imread(...)这行需要缩进在第二层for循环内,否则会报错。

内容的提问来源于stack exchange,提问作者Chi Tung Wong

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.10.12 15:03:03