使用LangChain LLMChain调用HuggingFacePipeline无输出问题排查

问题分析 #

你的代码中，直接调用hf("Who is the Pope ?")能正常返回结果，但通过LLMChain.run()却无输出，核心原因大概率是LangChain生成的完整prompt与模型预期格式不匹配，或是LLMChain未正确传递生成参数导致模型未生成有效内容。

解决方案 #

尝试以下几个调整步骤：

1. 优化Prompt模板格式 #

确保ASSISTANT:后添加空格，让模型清晰识别生成起始位置：

template = """SYSTEM: You are a helpful, respectful and honest INTP-T AI Assistant named Buddy. You are talking to a human User.
Always answer as helpfully and logically as possible, while being safe. Your answers should not include any harmful, political, religious, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
You like to use emojis. You can speak fluently in many languages, for example: English, Chinese.
You cannot access the internet, but you have vast knowledge, cutoff: 2021-09.
You are trained by OpenBuddy team, you are based on LLaMA and Falcon transformers model, not related to GPT or OpenAI.
USER: {question}
ASSISTANT: 
"""

2. 开启LLMChain调试日志 #

添加verbose=True可以查看LangChain生成的完整prompt，确认是否符合模型要求：

llm_chain = LLMChain(prompt=prompt, llm=hf, verbose=True)

3. 调整HuggingFacePipeline返回设置 #

创建HuggingFacePipeline时设置return_full_text=False，让模型只返回生成的回答部分（避免返回完整prompt导致LLMChain解析异常）：

hf = HuggingFacePipeline(pipeline=pipe, return_full_text=False)

修改后的完整代码 #

from langchain.llms import HuggingFacePipeline
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, BitsAndBytesConfig
from langchain import PromptTemplate, LLMChain
import torch

model_id  = "../models/openbuddy-llama2-34b-v11.1-bf16"  
tokenizer = AutoTokenizer.from_pretrained(model_id)
nf4_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_quant_type='nf4',
    bnb_4bit_use_double_quant=False,
    max_memory=24000  
)
model =  AutoModelForCausalLM.from_pretrained(
        model_id,
        quantization_config=nf4_config,
        )
pipe = pipeline(
    "text-generation", model=model, tokenizer=tokenizer, max_new_tokens=100, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id,
)
# 设置仅返回生成的回答内容
hf = HuggingFacePipeline(pipeline=pipe, return_full_text=False)

template = """SYSTEM: You are a helpful, respectful and honest INTP-T AI Assistant named Buddy. You are talking to a human User.
Always answer as helpfully and logically as possible, while being safe. Your answers should not include any harmful, political, religious, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
You like to use emojis. You can speak fluently in many languages, for example: English, Chinese.
You cannot access the internet, but you have vast knowledge, cutoff: 2021-09.
You are trained by OpenBuddy team, you are based on LLaMA and Falcon transformers model, not related to GPT or OpenAI.
USER: {question}
ASSISTANT: 
"""

prompt = PromptTemplate(template=template, input_variables=["question"])
# 开启调试日志查看prompt生成情况
llm_chain = LLMChain(prompt=prompt, llm=hf, verbose=True)

print(llm_chain.run("Who is the Pope ?"))

内容的提问来源于stack exchange，提问作者LazyTurtle