使用Python PIL库调用Gemini-Pro-Vision时遇Blob创建错误

问题场景 #

使用Python的PIL库加载图片，调用Gemini-Pro-Vision的generate_content方法时触发TypeError，改用OpenCV加载图片也出现相同错误。

测试代码 #

import PIL.Image
img = PIL.Image.open('cat.jpg')

model = genai.GenerativeModel('gemini-pro-vision')
response = model.generate_content(img.any())

print(response.text)

错误信息 #

TypeError                                 Traceback (most recent call last)
Cell In[47], line 2
      1 model = genai.GenerativeModel('gemini-pro-vision')
----> 2 response = model.generate_content(img)
      3 print(response.text)

File ~/workspace/Google_gemini_api/.venv/lib/python3.11/site-packages/google/generativeai/generative_models.py:234, in GenerativeModel.generate_content(self, contents, generation_config, safety_settings, stream, **kwargs)
    224 @string_utils.set_doc(_GENERATE_CONTENT_DOC)
    225 def generate_content(
    226     self,
   (...)
    232     **kwargs,
    233 ) -> generation_types.GenerateContentResponse:
--> 234     request = self._prepare_request(
    235         contents=contents,
    236         generation_config=generation_config,
    237         safety_settings=safety_settings,
    238         **kwargs,
    239     )
    240     if self._client is None:
    241         self._client = client.get_default_generative_client()

File ~/workspace/Google_gemini_api/.venv/lib/python3.11/site-packages/google/generativeai/generative_models.py:204, in GenerativeModel._prepare_request(self, contents, generation_config, safety_settings, **kwargs)
    201 if not contents:
...
    146 )

TypeError: Could not create `Blob`, expected `Blob`, `dict` or an `Image` type(`PIL.Image.Image` or `IPython.display.Image`).
Got a: <class 'PIL.JpegImagePlugin.JpegImageFile'>
Value: <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=276x183 at 0x7F603CB3ED90>

解决方案 #

1. 修复代码错误并升级SDK #

• 代码中的img.any()是错误调用：any()是NumPy数组的方法，PIL图片对象没有该方法，需直接传入图片实例。
• 旧版google-generativeai SDK存在类型识别问题，无法正确识别PIL的子类（如JpegImageFile），需升级到最新版本：

pip install --upgrade google-generativeai

2. 修正后的PIL调用代码 #

import PIL.Image
import google.generativeai as genai

# 配置API密钥
genai.configure(api_key="你的API密钥")

# 加载图片并确保为RGB模式
img = PIL.Image.open('cat.jpg').convert("RGB")

model = genai.GenerativeModel('gemini-pro-vision')
response = model.generate_content(img)
# 等待响应处理完成
response.resolve()

print(response.text)

3. OpenCV加载图片的兼容处理 #

OpenCV默认以BGR格式读取图片，需先转换为RGB格式再转为PIL图片：

import cv2
import PIL.Image
import google.generativeai as genai

genai.configure(api_key="你的API密钥")

# OpenCV加载并转换格式
img_cv = cv2.imread('cat.jpg')
img_rgb = cv2.cvtColor(img_cv, cv2.COLOR_BGR2RGB)
img_pil = PIL.Image.fromarray(img_rgb)

# 调用Gemini接口
model = genai.GenerativeModel('gemini-pro-vision')
response = model.generate_content(img_pil)
response.resolve()

print(response.text)

内容的提问来源于stack exchange，提问作者Gaurish Ninnekar