基于LangChain与FastAPI的AI聊天机器人后端：如何仅提取Bot回复

问题分析 #

返回结果包含完整模板和查询内容，主要原因有两点：

• 未遵循Mistral-7B-Instruct-v0.2模型要求的提示格式，导致模型输出包含了整个提示文本
• 未正确提取LLMChain生成的纯回答内容

解决方案 #

方法1：修正提示格式并提取生成文本（最优解） #

Mistral指令模型需要使用特定对话格式，能让模型精准生成回答，避免重复提示内容。修改代码如下：

template = """
<s>[INST]You are Bot, a AI bot to help user of my portfolio if they need to. Always be thankfull with the user from showing interest to my portfolio.
Depending on user's question you need to point them to the right section of the portfolio. 
If they want to contact me they can use the contact form or visit one of my social media via the links

My name is John Doe and this is my personal portfolio where I display my interests:
    - chess with my chess.com stats
    - running with my strava stats result
    - some pictures showing my accomplishments and my passion for travels and mountains
    - a contact form to reach me out
    - links to my different social media (facebook, linkedin, github, chess.com and strava)

User query : {question}[/INST]
"""

app = FastAPI()

@app.post("/conversation")
async def read_conversation(query:str):
    repo_id = "mistralai/Mistral-7B-Instruct-v0.2"

    llm1 = HuggingFaceHub(
        repo_id=repo_id, 
        model_kwargs={"temperature" : 0.7}
    )

    prompt = PromptTemplate(
        input_variables=["question"], template=template
    )
    chain = LLMChain(llm=llm1, prompt=prompt)
    response = chain.invoke({"question":query})

    # 直接提取模型生成的纯回答
    return {"response" : response["text"]}

方法2：正则表达式提取回答 #

如果暂时不想修改提示格式，可用正则从返回文本中提取Bot's answer :之后的内容：

import re

# 保留原模板和其他代码不变
@app.post("/conversation")
async def read_conversation(query:str):
    # ... 其他代码 ...
    response = chain.invoke({"question":query})
    # 提取Bot回答部分
    bot_answer = re.split(r"Bot's answer :", response["text"])[-1].strip()
    return {"response" : bot_answer}

方法3：使用LangChain OutputParser #

通过自定义解析器提取指定内容：

from langchain.output_parsers import RegexParser

# 定义解析规则
output_parser = RegexParser(
    regex=r"Bot's answer : (.*)",
    output_keys=["answer"]
)

# 绑定解析器到PromptTemplate
prompt = PromptTemplate(
    input_variables=["question"], 
    template=template,
    output_parser=output_parser
)

@app.post("/conversation")
async def read_conversation(query:str):
    # ... 其他代码 ...
    chain = LLMChain(llm=llm1, prompt=prompt)
    response = chain.invoke({"question":query})
    return {"response" : response["answer"]}

内容的提问来源于stack exchange，提问作者Mickael Maujean