如何正确加载本地LLM并在LangChain的initialize_agent函数中使用？

问题根源分析 #

flan-t5属于文本到文本生成模型，和OpenAI的补全类模型结构不同，默认的HuggingFacePipeline配置缺少必要的生成参数，且ZERO_SHOT_REACT_DESCRIPTION的默认prompt模板可能不匹配flan-t5的输出格式，导致Agent无法正确解析工具调用逻辑。

修正后的完整代码实现 #

1. 完整依赖导入 #

from langchain.agents import initialize_agent, AgentType
from langchain.tools import Tool
from langchain.llms import HuggingFacePipeline
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, pipeline

2. 正确加载本地模型并配置Pipeline #

def agent_lookup(name: str) -> str:
    # 加载本地模型与tokenizer
    model_id = llm_path  # 替换为你的本地模型路径
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

    # 配置文本生成pipeline，设置关键生成参数
    text_gen_pipeline = pipeline(
        "text2text-generation",
        model=model,
        tokenizer=tokenizer,
        max_new_tokens=200,
        temperature=0.1,  # 降低随机性，让输出逻辑更稳定
        top_p=0.95,
        repetition_penalty=1.15
    )

    # 包装为LangChain兼容的LLM实例
    llm = HuggingFacePipeline(pipeline=text_gen_pipeline)

    # 测试用工具函数
    def get_local_url(name: str) -> str:
        return f"local://{name}_resource"

    lookup_tool = Tool(
        name="Get local URL",
        func=get_local_url,
        description="Useful when you need to get the local URL corresponding to a given name. Input should be the target name string."
    )
    tools_for_agent = [lookup_tool]

    # 初始化Agent，自定义prompt适配flan-t5输出逻辑
    agent = initialize_agent(
        tools_for_agent,
        llm,
        agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
        verbose=True,
        handle_parsing_errors=True,
        agent_kwargs={
            "prefix": """Answer the following questions as best you can. You have access to the following tools:

{tools}

Use the following format strictly:

Question: the input question you must answer
Thought: you should always think about what to do next
Action: the action to take, must be one of [{tool_names}]
Action Input: the input to pass to the action
Observation: the result returned by the action
... (repeat Thought/Action/Action Input/Observation as needed)
Thought: I now have the final answer
Final Answer: the final answer to the original question

Begin!

Question: {input}
Thought:"""
        }
    )

    # 直接传入格式化后的字符串调用Agent
    prompt = f"Given the name {name} get me a local URL. Use Get local URL to solve"
    return agent.run(prompt)

3. 关键修正点说明 #

• 手动配置生成Pipeline：放弃from_model_id的默认配置，手动设置生成参数，确保模型输出长度足够、逻辑稳定。
• 优化工具描述：明确工具的输入要求，帮助模型生成符合格式的调用指令。
• 自定义Agent Prompt模板：ZERO_SHOT_REACT_DESCRIPTION的默认prompt适配补全类模型，自定义模板让flan-t5清晰理解思考-行动的流程规则。
• 调整调用方式：agent.run直接传入格式化后的字符串，避免Prompt对象解析带来的异常。

额外注意事项 #

• 确保本地模型文件完整，包含tokenizer配置和模型权重文件。
• 若仍出现解析错误，可进一步降低temperature减少随机性，或增大max_new_tokens确保输出包含完整的思考-行动链。
• flan-t5-large显存占用较高，建议在GPU环境运行，显存不足时可添加device_map="auto"参数加载模型。

内容的提问来源于stack exchange，提问作者juliocarrasquel