在Haystack中使用GoogleAIGeminiChatGenerator调用Gemini遇类型错误求助

我在Google Colab运行Haystack的Google AI集成教程时，前期步骤正常，但执行多轮对话代码时触发TypeError，错误来自Google的content_types.py的to_blob函数，即使未处理Blob场景也报错。尝试过修改消息添加方式、转换消息为Gemini预期的dict格式等方法，均未解决。

原测试代码 #

from haystack.utils import Secret
from haystack.dataclasses.chat_message import ChatMessage
from haystack_integrations.components.generators.google_ai import GoogleAIGeminiChatGenerator


gemini_chat = GoogleAIGeminiChatGenerator(model="gemini-pro", api_key=Secret.from_token("<MY_API_KEY>"))

messages = [ChatMessage.from_user("What is the most interesting thing you know?")]
res = gemini_chat.run(messages=messages)
for reply in res["replies"]:
    print(reply.content)

messages += res["replies"] + [ChatMessage.from_user("Tell me more about it")]
res = gemini_chat.run(messages=messages)
for reply in res["replies"]:
    print(reply.content)

报错信息 #

TypeError                                 Traceback (most recent call last)
<ipython-input-23-868854587ba9> in <cell line: 14>()
     12 
     13 messages += res["replies"] + [ChatMessage.from_user("Tell me more about it")]
---> 14 res = gemini_chat.run(messages=messages)
     15 for reply in res["replies"]:
     16     print(reply.content)

7 frames
/usr/local/lib/python3.10/dist-packages/google/generativeai/types/content_types.py in to_blob(blob)
    150                 "Could not recognize the intended type of the `dict`\n" "A content should have "
    151             )
---> 152         raise TypeError(
    153             "Could not create `Blob`, expected `Blob`, `dict` or an `Image` type"
    154             "(`PIL.Image.Image` or `IPython.display.Image`).\n"

TypeError: Could not create `Blob`, expected `Blob`, `dict` or an `Image` type(`PIL.Image.Image` or `IPython.display.Image`).
Got a: <class 'google.ai.generativelanguage_v1beta.types.content.Content'>
Value: parts {
  text: "What is the most interesting thing you know?"
}
role: "user"

解决方案 #

核心原因 #

Haystack的GoogleAIGeminiChatGenerator组件返回的replies中的消息对象，二次传入run方法时会被错误转换为Google原生的Content类型，而非Haystack标准的ChatMessage类型，导致类型不兼容报错。

修改后的代码 #

from haystack.utils import Secret
from haystack.dataclasses.chat_message import ChatMessage
from haystack_integrations.components.generators.google_ai import GoogleAIGeminiChatGenerator

gemini_chat = GoogleAIGeminiChatGenerator(model="gemini-pro", api_key=Secret.from_token("<MY_API_KEY>"))

# 第一轮对话
messages = [ChatMessage.from_user("What is the most interesting thing you know?")]
res = gemini_chat.run(messages=messages)
for reply in res["replies"]:
    print(reply.content)
    # 将返回的助手回复重新包装为Haystack ChatMessage对象
    messages.append(ChatMessage.from_assistant(reply.content))

# 第二轮对话
messages.append(ChatMessage.from_user("Tell me more about it"))
res = gemini_chat.run(messages=messages)
for reply in res["replies"]:
    print(reply.content)

关键修改点 #

• 避免直接拼接res["replies"]到消息列表，而是逐个取出回复内容，用ChatMessage.from_assistant()重新构造标准ChatMessage对象
• 确保消息列表中始终只包含Haystack的ChatMessage类型，避免混入Google原生类型

关于system角色的兼容问题 #

当前Haystack的Google Gemini集成组件对system角色支持存在兼容性问题，建议通过在用户消息中附带系统提示的方式替代：

messages = [
    ChatMessage.from_user("You are a professional, concise assistant. Answer the question: What is the most interesting thing you know?")
]

内容的提问来源于stack exchange，提问作者Bruce Nielson