timm调用RandomResizedCropAndInterpolation报TypeError排查

问题现象 #

参照timm官方教程编写数据增强代码扩充自有数据集样本时，代码运行抛出类型错误，核心报错信息为TypeError: 'builtin_function_or_method' object is not subscriptable，错误触发位置为timm库transforms模块get_params方法的图像面积计算逻辑。

原实现代码 #

import numpy as np
import torch
from PIL import Image
from timm.data.transforms_factory import create_transform
a = create_transform(224, is_training=True)
print(a)
pets_image_paths = './download.png'
image = Image.open(pets_image_paths)
# We can convert this into a tensor, and transpose the channels into the format that PyTorch expects:
np_image = np.array(image, dtype=np.float32)
image = torch.as_tensor(np_image).transpose(2, 0)[None]
from timm.data.transforms import RandomResizedCropAndInterpolation
tfm = RandomResizedCropAndInterpolation(size=350, interpolation='random')
import matplotlib.pyplot as plt
fig, ax = plt.subplots(2, 4, figsize=(10, 5))
for idx, im in enumerate([tfm(image) for i in range(4)]):
    ax[0, idx].imshow(im)
for idx, im in enumerate([tfm(image) for i in range(4)]):
    ax[1, idx].imshow(im)
fig.tight_layout()
plt.show()

完整异常栈 #

Traceback (most recent call last):
  File "/home/cvpr/PycharmProjects/timm_tutorials/9_augmentation.py", line 24, in <module>
    for idx, im in enumerate([tfm(image) for i in range(4)]):
  File "/home/cvpr/PycharmProjects/timm_tutorials/9_augmentation.py", line 24, in <listcomp>
    for idx, im in enumerate([tfm(image) for i in range(4)]):
  File "/home/cvpr/anaconda3/envs/timm_tutorials/lib/python3.8/site-packages/timm/data/transforms.py", line 181, in __call__
    i, j, h, w = self.get_params(img, self.scale, self.ratio)
  File "/home/cvpr/anaconda3/envs/timm_tutorials/lib/python3.8/site-packages/timm/data/transforms.py", line 143, in get_params
    area = img.size[0] * img.size[1]
TypeError: 'builtin_function_or_method' object is not subscriptable

问题原因 #

RandomResizedCropAndInterpolation变换的输入要求为PIL Image对象，不支持直接传入PyTorch张量：

• 对PIL Image对象而言，.size是存储图像(宽度, 高度)的元组属性，支持下标索引取值
• 对PyTorch Tensor对象而言，.size是用来获取张量形状的内置方法，不是可下标索引的属性，直接对方法对象做[0]索引就会抛出上述类型错误

原代码错误地提前将PIL读取的图像转换为了带batch维度的PyTorch张量，原PIL图像对象还被张量值覆盖，最终传入增强变换的是不符合输入要求的张量，触发报错。

修复方法 #

• 移除提前将PIL图像转为PyTorch张量的逻辑，直接将PIL读取的原始图像传入增强变换
• 所有针对PIL图像的增强操作执行完成后，再根据需要添加ToTensor()、归一化等转换为张量的操作
• 可视化时注意：matplotlib的imshow方法接收HWC格式的数组/PIL图像，若后续转成了CHW格式的张量，需要先调整维度顺序再传入显示

修复后可运行代码 #

from PIL import Image
from timm.data.transforms import RandomResizedCropAndInterpolation
import matplotlib.pyplot as plt

# 直接读取为PIL Image，不提前转张量
pets_image_paths = './download.png'
image = Image.open(pets_image_paths)

tfm = RandomResizedCropAndInterpolation(size=350, interpolation='random')
fig, ax = plt.subplots(2, 4, figsize=(10, 5))

# 传入PIL图像做增强
for idx, im in enumerate([tfm(image) for i in range(4)]):
    ax[0, idx].imshow(im)
for idx, im in enumerate([tfm(image) for i in range(4)]):
    ax[1, idx].imshow(im)
    
# 隐藏坐标轴优化显示效果
for row in ax:
    for col in row:
        col.axis('off')
fig.tight_layout()
plt.show()

内容的提问来源于stack exchange，提问作者Khawar Islam