自定义图像数据集Reshape出错，如何生成MNIST格式张量？

Got it, let's break down why you're hitting that error and get your data into the exact shape you need, step by step.

First, the root of the ValueError is simple math: your current training images are stored as single-channel (grayscale) data (each image is 32×32=1024 pixels), but you're trying to reshape them into a 3-channel (RGB) format. The numbers don't add up: 2758 × 32 × 32 × 3 = 8,472,576 total elements, but your current array only has 2758 × 1024 = 2,824,192 elements. That's why the reshape fails.

We'll fix both the image channel issue and your label shape to hit your target format.

1. Fix the Image Channel Problem #

The issue comes from your pickling code: you're only extracting the first value of each pixel (which strips RGB images down to grayscale, or leaves grayscale as-is). Let's adjust this based on your actual image type:

If your original images are RGB (3-channel) #

Replace this line in your pickling code:

# Old code: drops 2 channels, keeps only the first
pixels = [f[0] for f in list(img.getdata())]

With this:

# Keep all 3 channels, flatten into a 1D list (R1,G1,B1,R2,G2,B2...)
pixels = [val for pixel in list(img.getdata()) for val in pixel]

This makes each image's pixel count 32×32×3=3072, so your train_images will load as (2758, 3072) — which can safely be reshaped to (2758, 32, 32, 3).

If your original images are grayscale, but you need 3-channel (simulated RGB) #

If you want to convert single-channel grayscale into 3-channel (repeating the grayscale value for R/G/B), modify the pixel extraction code like this:

# Old code
pixels = [f[0] for f in list(img.getdata())]
# New code: repeat each grayscale value 3 times to mimic RGB
pixels = []
for gray_val in [f[0] for f in list(img.getdata())]:
    pixels.extend([gray_val, gray_val, gray_val])

This also gives you 3072 pixels per image, making the reshape work perfectly.

2. Fix the Label Shape (from (n,) to (n,1)) #

Once you load your labels, converting them to the (n,1) shape is super straightforward with NumPy:

import numpy as np

# Option 1: Use reshape (flexible for any size)
train_labels = train_labels.reshape(-1, 1)
valid_labels = valid_labels.reshape(-1, 1)
test_labels = test_labels.reshape(-1, 1)

# Option 2: Use np.newaxis (concise alternative)
train_labels = train_labels[:, np.newaxis]

Either method will turn your (2758,) labels into the (2758,1) shape you want.

3. Verify the Final Shape #

After re-generating your pkl file with the modified pickling code, load it and check the shapes to confirm:

print(train_images.shape)  # Should output (2758, 3072)
train_images = train_images.reshape(train_images.shape[0], 32, 32, 3)
print(train_images.shape)  # Now shows (2758, 32, 32, 3)
print(train_labels.shape)  # Shows (2758, 1)

Bonus: Normalize Your Image Data (Recommended for CNNs) #

Once reshaped, it's standard practice to normalize pixel values to the 0-1 range to help your CNN train better:

train_images = train_images.astype('float32') / 255.0
valid_images = valid_images.astype('float32') / 255.0
test_images = test_images.astype('float32') / 255.0

内容的提问来源于stack exchange，提问作者Park Yujin