LangChain调用LLMChain：如何避免输入混入输出致响应截断？

问题根源 #

Mistral-7B-Instruct-v0.2是经过指令微调的对话模型，要求输入遵循特定的[INST] 指令 [/INST]格式。未按该格式封装的prompt会被模型误判为需要重复输出的上下文内容，同时默认生成token数限制可能导致响应提前截断。

解决方案 #

1. 适配Mistral的指令格式 #

将prompt内容包裹在Mistral要求的[INST]和[/INST]标签内，明确区分指令与模型输出区域：

from langchain.llms import HuggingFaceHub
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain

llm_hf = HuggingFaceHub(
    repo_id="mistralai/Mistral-7B-Instruct-v0.2",
    model_kwargs={"temperature": 0.9, "max_new_tokens": 200}  # 增加token额度避免截断
)

# 适配Mistral专属指令格式
duck_specialist_template = """
[INST]
I want you to act as a specialist in ducks, covering aspects from their biology to conservation.

Your role involves providing expert insights into different duck species, including their habitats, behaviors, and the challenges they face in their natural environments. Additionally, you're tasked with advising on conservation strategies to protect these species and their habitats from threats such as habitat loss, pollution, and climate change.

For any inquiry about ducks, offer a comprehensive overview that includes species identification, habitat preferences, dietary habits, migratory patterns, and conservation status. Moreover, suggest practical conservation measures that can be implemented to support the health and sustainability of duck populations.

Can you provide funny name suggestions on a duck that looks like {duck_description}?
[/INST]
"""

tiny = "a small, energetic duck with bright blue feathers and a penchant for doing somersaults in the water. Its quack is surprisingly deep for its size"

prompt_template = PromptTemplate(
    input_variables=["duck_description"],
    template=duck_specialist_template,
)

chain = LLMChain(llm=llm_hf, prompt=prompt_template)
print(chain.run(duck_description=tiny))

2. 调整生成参数 #

在model_kwargs中设置max_new_tokens（建议200及以上），确保模型有足够的token额度生成完整响应，避免截断。

3. 推荐使用Chat模型适配类（更优雅方案） #

使用ChatHuggingFace配合ChatPromptTemplate，自动适配对话类模型的格式要求：

from langchain.chat_models import ChatHuggingFace
from langchain.prompts.chat import ChatPromptTemplate, HumanMessagePromptTemplate

chat_model = ChatHuggingFace(
    llm=HuggingFaceHub(
        repo_id="mistralai/Mistral-7B-Instruct-v0.2",
        model_kwargs={"temperature": 0.9, "max_new_tokens": 200}
    )
)

human_prompt = HumanMessagePromptTemplate.from_template("""
I want you to act as a specialist in ducks, covering aspects from their biology to conservation.

Your role involves providing expert insights into different duck species, including their habitats, behaviors, and the challenges they face in their natural environments. Additionally, you're tasked with advising on conservation strategies to protect these species and their habitats from threats such as habitat loss, pollution, and climate change.

For any inquiry about ducks, offer a comprehensive overview that includes species identification, habitat preferences, dietary habits, migratory patterns, and conservation status. Moreover, suggest practical conservation measures that can be implemented to support the health and sustainability of duck populations.

Can you provide funny name suggestions on a duck that looks like {duck_description}?
""")

chat_prompt = ChatPromptTemplate.from_messages([human_prompt])
chain = chat_prompt | chat_model
print(chain.invoke({"duck_description": tiny}).content)

效果说明 #

修改后模型会正确识别指令区域，不再重复输出prompt内容，同时max_new_tokens参数确保响应有足够长度完成生成。

内容的提问来源于stack exchange，提问作者StatSid