如何在LangChain中结合ConversationalRetrievalQAChain、Agent与SerpAPI工具

需求概述 #

已分别实现两个独立功能：

• 基于ConversationalRetrievalQAChain+OpenAI嵌入+Chroma向量库的产品PDF文档检索，用于回答产品特性、技术规格类问题
• 基于SerpAPI的联网搜索，用于获取产品最新定价信息
  需要将两者整合，实现：

1. 自动判断用户问题是否需要联网搜索
2. 价格类问题调用SerpAPI，其他产品相关问题调用本地文档检索
3. 对话历史在两种场景中共享，支持指代类问题（如“它的价格是多少？”）的识别

实现方案 #

通过LangChain的Agent机制实现路由：将ConversationalRetrievalQAChain包装为自定义工具，与SerpAPI工具一起交给ChatAgent管理，由LLM根据问题类型选择对应工具；同时统一处理对话历史的传递，确保上下文连贯性。

整合后的完整代码 #

// 导入所需依赖
import { RequestHandler } from '@sveltejs/kit';
import { LangChainStream, CallbackManager } from 'langchain/server';
import { OpenAIEmbeddings, ChatOpenAI } from 'langchain/chat_models/openai';
import { Chroma } from 'langchain/vectorstores/chroma';
import { ConversationalRetrievalQAChain } from 'langchain/chains';
import { PromptTemplate } from 'langchain/prompts';
import { SerpAPI } from 'langchain/tools';
import { ChatAgent, AgentExecutor } from 'langchain/agents';
import { Message } from './types'; // 确保导入你的Message类型定义

// 问题改写Prompt
const CONDENSE_PROMPT = `Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question.

Chat History:
{chat_history}

Follow Up Input: {question}

Standalone question:`;

// 文档检索QA Prompt
const QA_PROMPT = `You are a helpful AI assistant for sales reps to answer questions about product features and technical specifications.
Use the following pieces of context to answer the question at the end.
If you don't know the answer, just say you don't know. DO NOT try to make up an answer.
If the question is not related to the context, politely respond that you are tuned to only answer questions that are related to the context.

{context}

Question: {question}
Helpful answer:`;

export const POST: RequestHandler = async ({ request }) => {
  const { messages } = await request.json();
  const { stream, handlers } = LangChainStream();
  const callbacks = CallbackManager.fromHandlers(handlers);

  const openAIApiKey = OPENAI_API_KEY;
  const embeddings = new OpenAIEmbeddings({ openAIApiKey });

  // 初始化向量库
  const store = await Chroma.fromExistingCollection(embeddings, {
    collectionName: 'langchain',
  });

  // 初始化LLM模型
  const streamingModel = new ChatOpenAI({
    openAIApiKey,
    temperature: 0,
    streaming: true,
    callbacks,
  });

  const nonStreamingModel = new ChatOpenAI({
    openAIApiKey,
    temperature: 0,
  });

  // 构建文档检索链
  const retrievalChain = ConversationalRetrievalQAChain.fromLLM(
    streamingModel,
    store.asRetriever(),
    {
      returnSourceDocuments: true,
      verbose: false,
      qaChainOptions: {
        type: "stuff",
        prompt: PromptTemplate.fromTemplate(QA_PROMPT)
      },
      questionGeneratorChainOptions: {
        template: CONDENSE_PROMPT,
        llm: nonStreamingModel,
      },
    }
  );

  // 将检索链包装为自定义工具
  const retrievalTool = {
    name: "product-docs-retrieval",
    description: "用于回答产品特性、技术规格等非价格类问题，基于本地产品PDF文档检索",
    async func({ question, chat_history }: { question: string, chat_history: string }) {
      const result = await retrievalChain.call({
        question,
        chat_history
      });
      return result.text;
    }
  };

  // 初始化SerpAPI工具
  const serpTool = new SerpAPI(SERPAPI_API_KEY, {
    location: "Austin,Texas,United States",
    hl: "en",
    gl: "us",
  });

  // 定义Agent使用的工具列表
  const tools = [retrievalTool, serpTool];

  // 自定义Agent Prompt，明确工具适用场景
  const agentPrompt = ChatAgent.createPrompt(tools, {
    prefix: `你是销售代表的AI助手，负责回答产品相关问题。请根据问题类型选择合适的工具：
- 如果是价格、最新市场信息类问题，使用SerpAPI工具
- 如果是产品特性、技术规格类问题，使用product-docs-retrieval工具
请始终结合对话历史理解用户的问题，比如用户问"它的价格是多少？"时，要根据历史对话确定指代的产品。`,
    suffix: `开始处理用户问题：
{chat_history}
用户问题：{input}
{agent_scratchpad}`
  });

  // 创建Agent和执行器
  const agent = ChatAgent.fromLLMAndTools(streamingModel, tools, {
    prompt: agentPrompt,
  });

  const executor = AgentExecutor.fromAgentAndTools({
    agent,
    tools,
    verbose: false,
  });

  // 处理对话历史格式
  const chatHistory = messages.slice(0, -1).map((m: Message) => `${m.role}: ${m.content}`).join('\n');
  const latestQuestion = (messages as Message[]).at(-1)!.content;

  // 调用Agent执行器
  executor.call({
    input: latestQuestion,
    chat_history: chatHistory
  }).catch(console.error);

  return new StreamingTextResponse(stream);
};

关键说明 #

1. 工具包装：将ConversationalRetrievalQAChain包装为符合LangChain工具规范的自定义函数，明确其适用场景，让Agent能正确识别调用时机
2. 对话历史处理：统一将对话历史传递给Agent和检索链，确保指代类问题能结合上下文理解
3. Agent Prompt引导：通过自定义Prompt明确两种工具的分工，帮助LLM做出正确的工具选择决策
4. 流式输出兼容：保留原有的流式响应配置，确保用户体验一致

内容的提问来源于stack exchange，提问作者Johannes Fahrenkrug