Stable Diffusion图像变体代码报错：CLIPVisionModelWithProjection无get_image_features属性

我正在基于Stable Diffusion图像变体示例开发，严格遵循步骤编写代码后出现错误。

代码 #

from pathlib import Path
from lambda_diffusers import StableDiffusionImageEmbedPipeline

from PIL import Image
import torch

#device = "cuda" if torch.cuda.is_available() else "cpu"
device = "cpu"
pipe = StableDiffusionImageEmbedPipeline.from_pretrained("lambdalabs/sd-image-variations-diffusers")
pipe = pipe.to(device)

im = Image.open("/home/cvpr/Desktop/bird.jpg")
num_samples = 4
image = pipe(num_samples*[im], guidance_scale=3.0)
image = image["sample"]

base_path = Path("outputs/im2im")
base_path.mkdir(exist_ok=True, parents=True)
for idx, im in enumerate(image):
    im.save(base_path/f"{idx:06}.jpg")

报错回溯 #

Traceback (most recent call last):
  File "/media/cvpr/CM_1/lambda-diffusers/image_variation.py", line 46, in <module>
    image = pipe(num_samples*[im], guidance_scale=3.0)
  File "/media/cvpr/CM_1/lambda-diffusers/.venv/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/media/cvpr/CM_1/lambda-diffusers/lambda_diffusers/pipelines/pipeline_stable_diffusion_im_embed.py", line 77, in __call__
    image_embeddings = self.image_encoder.get_image_features(**input_image)
  File "/media/cvpr/CM_1/lambda-diffusers/.venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'CLIPVisionModelWithProjection' object has no attribute 'get_image_features'

解决方法 #

错误原因是CLIPVisionModelWithProjection类并没有get_image_features这个方法，正确获取图像特征的方式是直接调用模型并提取输出的image_embeds。

找到lambda_diffusers/pipelines/pipeline_stable_diffusion_im_embed.py文件的第77行，将原代码：

image_embeddings = self.image_encoder.get_image_features(**input_image)

替换为：

outputs = self.image_encoder(**input_image)
image_embeddings = outputs.image_embeds

修改后重新运行代码即可解决该报错。

内容的提问来源于stack exchange，提问作者Khawar Islam