如何用Python、OpenCV、Keras修复Conv2D输入形状维度不匹配问题

我正在学习计算机视觉，在MacBook上使用Python、OpenCV、Keras开发一款针对模糊、含噪、失焦图像的去噪锐化程序。训练模型时使用了Conv2D层，model.fit阶段可正常运行，但调用model.predict时出现维度不匹配错误，终端提示“期望最小维度为4，实际得到维度3，接收的完整形状为(None,64,64)”。我具备应用数学背景，知晓矩阵维度不匹配的概念，但作为深度学习新手，需要解决该问题的帮助。

终端错误输出 #

$ python sharpen_denoise.py
2023-09-12 18:25:03.054961: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
Epoch 1/5
1/1 [==============================] - 1s 699ms/step - loss: 0.0543
Epoch 2/5
1/1 [==============================] - 0s 58ms/step - loss: 0.0532
Epoch 3/5
1/1 [==============================] - 0s 50ms/step - loss: 0.0524
Epoch 4/5
1/1 [==============================] - 0s 52ms/step - loss: 0.0516
Epoch 5/5
1/1 [==============================] - 0s 60ms/step - loss: 0.0504
Traceback (most recent call last):
  File "sharpen_denoise.py", line 59, in <module>
    output_image = denoise_and_sharpen_image(model, noisy_image)
  File "sharpen_denoise.py", line 52, in denoise_and_sharpen_image
    denoised_image = model.predict(noisy_image)
  File "/Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/keras/engine/training.py", line 1751, in predict
    tmp_batch_outputs = self.predict_function(iterator)
  File "/Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 885, in __call__
    result = self._call(*args, **kwds)
  File "/Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 933, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "/Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 760, in _initialize
    *args, **kwds))
  File "/Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 3066, in _get_concrete_function_internal_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)
  File "/Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 3463, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 3308, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 1007, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 668, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 994, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

    /Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/keras/engine/training.py:1586 predict_function  *
        return step_function(self, iterator)
    /Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/keras/engine/training.py:1576 step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    /Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/distribute/distribute_lib.py:1286 run
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/distribute/distribute_lib.py:2849 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/tensorflow/python/distribute/distribute_lib.py:3632 _call_for_each_replica
        return fn(*args, **kwargs)
    /Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/keras/engine/training.py:1569 run_step  **
        outputs = model.predict_step(data)
    /Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/keras/engine/training.py:1537 predict_step
        return self(x, training=False)
    /Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/keras/engine/base_layer.py:1020 __call__
        input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
    /Users/infinite/opt/anaconda3/envs/opencv-env/lib/python3.6/site-packages/keras/engine/input_spec.py:234 assert_input_compatibility
        str(tuple(shape)))

    ValueError: Input 0 of layer sequential is incompatible with the layer: : expected min_ndim=4, found ndim=3. Full shape received: (None, 64, 64)

完整代码 #

import numpy as np
import cv2 as cv
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers


# Create and Compile Deep Learning Model

def create_model(input_shape):
    model = keras.Sequential()

    # Encoder
    model.add(layers.Input(shape=input_shape))
    model.add(layers.Conv2D(64, (3, 3), activation='relu', padding='same'))
    model.add(layers.Conv2D(64, (3, 3), activation='relu', padding='same'))
    model.add(layers.MaxPooling2D((2, 2), padding='same'))

    # decoder
    model.add(layers.Conv2D(64, (3, 3), activation='relu', padding='same'))
    model.add(layers.Conv2D(64, (3, 3), activation='relu', padding='same'))
    model.add(layers.UpSampling2D((2, 2)))
    model.add(layers.Conv2D(1, (3, 3), activation='sigmoid', padding='same'))

    return model


input_shape = (256, 256, 1)  # Adjust input_shape to size of image
model = create_model(input_shape)
model.compile(optimizer='adam', loss='mse')


# Load and Preprocess Noisy Image
def load_and_preprocess_image(image_path):
    image = cv.imread(image_path, cv.IMREAD_GRAYSCALE)
    image = cv.resize(image, input_shape[:2])
    image = image / 255.0  # Normalize Pixel Values to [0,1]
    image = np.expand_dims(image, axis=0)  # Add Batch Dimension
    return image


noisy_image = load_and_preprocess_image(
    '/Users/infinite/Desktop/PythonPractice/openCV/photos/Noisy_Image.jpg')  # Load Image Path

# Train the Model
model.fit(tf.expand_dims(noisy_image, axis=-1), noisy_image, epochs=5)
#model.fit(noisy_image, noisy_image, epochs=100, batch_size=1)


# Use Trained Model To Sharpen and DeBlur
def denoise_and_sharpen_image(model, noisy_image):
    denoised_image = model.predict(noisy_image)
    denoised_image = denoised_image.squeeze()  # Remove the Batch Dimension
    denoised_image = cv.normalize(denoised_image, None, 0, 255, cv.NORM_MINMAX)
    denoised_image = denoised_image.astype(np.uint8)
    return denoised_image


output_image = denoise_and_sharpen_image(model, noisy_image)

# Display The Noisy and Denoised Images
cv.imshow('Noisy Image', noisy_image.squeeze())
cv.imshow('Denoised_Image', output_image)

cv.waitKey(0)
cv.destroyAllWindows()

解决方案 #

问题根源 #

你的模型输入定义为(256,256,1)，要求输入是4维张量：(批量数, 高度, 宽度, 通道数)。但训练时你手动给输入加了通道维度（tf.expand_dims(noisy_image, axis=-1)），而预测时直接传入的noisy_image只有3维（(1,256,256)），导致维度不匹配。

修复步骤 #

1. 统一预处理流程，添加通道维度
   修改load_and_preprocess_image函数，加载灰度图后直接添加通道维度，确保输出是4维张量：

   def load_and_preprocess_image(image_path):
       image = cv.imread(image_path, cv.IMREAD_GRAYSCALE)
       image = cv.resize(image, input_shape[:2])
       image = image / 255.0  # 归一化像素值到[0,1]
       image = np.expand_dims(image, axis=-1)  # 添加通道维度（灰度图通道数为1）
       image = np.expand_dims(image, axis=0)  # 添加批量维度
       return image
   

2. 修正训练时的输入
   预处理后的数据已经符合模型要求的4维格式，不需要再额外扩展维度，修改model.fit行：

   model.fit(noisy_image, noisy_image, epochs=5)
   

3. 验证预测维度
   现在调用model.predict(noisy_image)时，输入是标准的4维张量，与模型输入要求一致，不会再出现维度不匹配错误。

补充说明 #

Keras的Conv2D层强制要求输入为4维张量，其中：

• 批量数：一次输入的样本数量，这里是1
• 高度/宽度：图像的尺寸，对应你的256x256
• 通道数：灰度图为1，彩色RGB图为3

训练和预测时的输入维度必须严格匹配模型定义的input_shape，否则会触发维度不匹配错误。

内容的提问来源于stack exchange，提问作者BobbyG