使用Keras预训练Xception模型预测图片时遇类型转换错误求助

触发的错误信息：

ValueError: Attempt to convert a value (<PIL.Image.Image image mode=RGB size=299x299 at 0x7F1DF6044F10>)
with an unsupported type (<class 'PIL.Image.Image'>) to a Tensor.

原代码如下：

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import matplotlib.pyplot as plt
# Load the pre-trained Xception model to be used as the base encoder.
xception = keras.applications.Xception(
    include_top=False, weights="imagenet", pooling="avg"
)
# Set the trainability of the base encoder.
for layer in xception.layers:
 layer.trainable = False
# Receive the images as inputs.
#inputs = layers.Input(shape=(299, 299, 3), name="image_input")

input ='/content/1.png'
input = tf.keras.preprocessing.image.load_img(input,target_size=(299,299,3))

BATCH_SIZE = 1
NUM_BOXES = 5
IMAGE_HEIGHT = 256
IMAGE_WIDTH = 256
CHANNELS = 3
CROP_SIZE = (24, 24)


boxes = tf.random.uniform(shape=(NUM_BOXES, 4))
box_indices = tf.random.uniform(shape=(NUM_BOXES,), minval=0,
maxval=BATCH_SIZE, dtype=tf.int32)
output = tf.image.crop_and_resize(input, boxes, box_indices, CROP_SIZE)
xception_input = tf.keras.applications.xception.preprocess_input(output)
plt.imshow(xception_input/255.)

问题原因与修复方案 #

核心问题 #

tf.image.crop_and_resize要求输入必须是Tensor类型，但你用tf.keras.preprocessing.image.load_img加载得到的是PIL Image对象，TensorFlow无法直接将其转换为Tensor，因此触发错误。

修复步骤 #

1. 将PIL Image转为Tensor：用tf.keras.preprocessing.image.img_to_array或者tf.convert_to_tensor完成格式转换。
2. 添加batch维度：crop_and_resize要求输入形状为(batch_size, height, width, channels)，单张图片需要用tf.expand_dims添加batch维度。
3. 调整可视化代码：crop_and_resize输出是(5,24,24,3)的Tensor（对应5个裁剪区域），plt.imshow只能显示单张图片，需要指定显示其中一个区域，或者用子图展示所有裁剪结果。

修复后的完整代码 #

import tensorflow as tf
from tensorflow import keras
import matplotlib.pyplot as plt

# 加载预训练Xception模型
xception = keras.applications.Xception(
    include_top=False, weights="imagenet", pooling="avg"
)
# 冻结模型层
for layer in xception.layers:
    layer.trainable = False

# 加载图片并转换为Tensor，添加batch维度
img_path = '/content/1.png'
# 加载为PIL Image
img_pil = tf.keras.preprocessing.image.load_img(img_path, target_size=(299, 299))
# 转为Tensor并添加batch维度
input_tensor = tf.expand_dims(tf.keras.preprocessing.image.img_to_array(img_pil), axis=0)

BATCH_SIZE = 1
NUM_BOXES = 5
CROP_SIZE = (24, 24)

# 生成随机裁剪框（注意：crop_and_resize的boxes坐标是[ymin, xmin, ymax, xmax]，范围0-1）
boxes = tf.random.uniform(shape=(NUM_BOXES, 4), minval=0, maxval=1)
box_indices = tf.random.uniform(shape=(NUM_BOXES,), minval=0, maxval=BATCH_SIZE, dtype=tf.int32)

# 执行裁剪
output = tf.image.crop_and_resize(input_tensor, boxes, box_indices, CROP_SIZE)
# 预处理Xception输入
xception_input = tf.keras.applications.xception.preprocess_input(output)

# 可视化第一个裁剪区域
plt.imshow(xception_input[0]/255.)
plt.show()

# 如果要展示所有5个裁剪区域，可用子图：
# fig, axes = plt.subplots(1, 5, figsize=(15,3))
# for i in range(NUM_BOXES):
#     axes[i].imshow(xception_input[i]/255.)
#     axes[i].axis('off')
# plt.show()

内容的提问来源于stack exchange，提问作者Jacob