关于LangChain中LLM无法读取HumanMessage的id及additional_kwargs的确认请求

两种方案的验证结果 #

• 方案1：设置HumanMessage的id为标识符
  虽能在LangSmith中看到id字段，但LLM无法识别这些id，无法基于此完成用户ID统计任务。
• 方案2：将标识符放入additional_kwargs的uid字段
  同样可在LangSmith中观测到该字段，但LLM仍无法读取uid信息，返回内容称未发现相关结构化数据。

验证代码 #

memory = ConversationBufferMemory(return_messages=True)
mem_vars = memory.load_memory_variables({})
pretty_print("Memory Variables init", mem_vars)
pretty_print("Memory Variables in str list (buffer_as_str) init", memory.buffer_as_str)

memory.buffer.append(AIMessage(content="This is a Gaming Place"))
mem_vars = memory.load_memory_variables({})
pretty_print("Memory Variables seeded", mem_vars)
pretty_print(
    "Memory Variables in str list (buffer_as_str), seeded", memory.buffer_as_str
)

memory.buffer.append(HumanMessage(content="Hello dudes", id="user-1"))
memory.buffer.append(HumanMessage(content="hi", id="user-2"))
memory.buffer.append(HumanMessage(content="yo yo", id="user-3"))
memory.buffer.append(HumanMessage(content="nice to see you", id="user-4"))
memory.buffer.append(HumanMessage(content="hoho dude", id="user-5"))
memory.buffer.append(HumanMessage(content="o lalala", id="user-L"))
memory.buffer.append(HumanMessage(content="guten tag", id="user-XXXXL"))
memory.buffer.append(HumanMessage(content="Let's get started, ok?", id="user-1"))
memory.buffer.append(HumanMessage(content="YES", id="user-2"))
memory.buffer.append(HumanMessage(content="YEAH....", id="user-3"))
memory.buffer.append(HumanMessage(content="Cool..", id="user-4"))
memory.buffer.append(HumanMessage(content="yup.", id="user-5"))
memory.buffer.append(HumanMessage(content="Great.....", id="user-L"))
memory.buffer.append(HumanMessage(content="alles klar", id="user-XXXXL"))
memory.buffer.append(HumanMessage(content="Opppsssssss.", id="user-5"))
mem_vars = memory.load_memory_variables({})
pretty_print("Memory Variables", mem_vars)
pretty_print("Memory Variables in str list (buffer_as_str)", memory.buffer_as_str)


def convert_memory_to_dict(memory: ConversationBufferMemory) -> List[Dict[str, str]]:
    """Convert the memory to the dict, role is id, content is the message content."""
    res = [
        """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. 
If the AI does not know the answer to a question, it truthfully says it does not know.

Notice: The 'uid' is user-id, 'role' is user role for human or ai, 'content' is the message content.

"""
    ]
    history = memory.load_memory_variables({})["history"]
    for hist_item in history:
        role = "human" if isinstance(hist_item, HumanMessage) else "ai"
        res.append(
            {
                "role": role,
                "content": hist_item.content,
                "uid": hist_item.id if role == "human" else "",
            }
        )
    return res


cxt_dict = convert_memory_to_dict(memory)
pretty_print("cxt_dict", cxt_dict)


def build_chain_without_parsing(
    model: BaseChatModel,
) -> RunnableSerializable[Dict, str]:
    prompt = ChatPromptTemplate.from_messages(
        [
            SystemMessage(
                content=("You are an AI assistant." "You can handle the query of user.")
            ),
            MessagesPlaceholder(variable_name="history"),
            HumanMessagePromptTemplate.from_template("{query}"),
        ]
    )
    return (
        prompt | model
    )  # comment model, you can see the filled template after invoking the chain.


model = llm
human_query = HumanMessage(
    """Count the number of 'uid'.""",
    id="user-X",
)

res = build_chain_without_parsing(model).invoke(
    {
        "history": cxt_dict,
        "query": human_query,
    }
)
pretty_print("Result", res)

LLM返回结果（中文翻译） #

你似乎想根据之前描述的对话结构统计唯一的'uid'值数量，但你提供的对话片段里没有明确的'uid'值，也没有包含'uid'、'role'和'content'字段的结构化格式。对话只是一系列问候和确认内容，没有可用于统计唯一用户ID('uid')的结构化数据或标识符。

如果你有包含'uid'、'role'和'content'字段的特定数据集或条目列表，请提供这些数据，我就能帮你统计其中唯一'uid'值的数量。

结论 #

你的理解完全正确：这两种方式都无法让LLM读取到用户标识符信息。原因是LangChain在将消息传递给LLM时，默认只会把消息的content和角色信息纳入prompt的可见内容，id和additional_kwargs中的额外字段不会被自动注入到LLM能感知的上下文里。

内容的提问来源于stack exchange，提问作者TeeTracker