LangChain自定义LLM接入RetrievalQA链失败求助

正在构建生产级RAG聊天机器人，已拥有自研LLM API，通过自定义LLM类集成到LangChain后，单独调用_call方法能正常返回结果，但将其用于RetrievalQA.from_chain_type时报错，提示"Encountered some errors. Please recheck your request!"。已排除检索器问题，且API上下文长度问题已修复，仍无法解决。

自定义LLM类代码：

from pydantic import Extra
import requests
from typing import Any, List, Mapping, Optional

from langchain.callbacks.manager import CallbackManagerForLLMRun
from langchain.llms.base import LLM

class LlamaLLM(LLM):
    llm_url = 'https:/myhost/llama/api'

    class Config:
        extra = Extra.forbid

    @property
    def _llm_type(self) -> str:
        return "Llama2 7B"

    def _call(
        self,
        prompt: str,
        stop: Optional[List[str]] = None,
        run_manager: Optional[CallbackManagerForLLMRun] = None,
        **kwargs: Any,
    ) -> str:
        if stop is not None:
            raise ValueError("stop kwargs are not permitted.")

        payload = {
            "inputs": prompt,
            "parameters": {"max_new_tokens": 100},
            "token": "abcdfejkwehr"
        }

        headers = {"Content-Type": "application/json"}

        response = requests.post(self.llm_url, json=payload, headers=headers, verify=False)
        response.raise_for_status()

        return response.json()['generated_text']

    @property
    def _identifying_params(self) -> Mapping[str, Any]:
        return {"llmUrl": self.llm_url}

单独测试代码（可正常运行）：

llm = LlamaLLM()
prompt = "[INST] Question: Who is Albert Einstein? \n Answer: [/INST]"
result = llm._call(prompt)
# 返回结果：Albert Einstein (1879-1955) was a German-born theoretical physicist...

QA链构建及报错代码：

from langchain.prompts import PromptTemplate
from langchain.chains import RetrievalQA

template = """[INST] <<SYS>>

Answer the question base on the context below.

<</SYS>>

Context: {context}
Question: {question}
Answer:
[/INST]"""
QA_CHAIN_PROMPT = PromptTemplate(input_variables=["context", "question"],template=template,)

qa_chain = RetrievalQA.from_chain_type(llm,
                                       verbose=True,
                                       retriever=custom_retriever,
                                       return_source_documents=True,
                                       chain_type_kwargs={"prompt": QA_CHAIN_PROMPT})

question = "Is probability a class topic?"
result = qa_chain({"query": question})
# 报错：Encountered some errors. Please recheck your request!

额外提示：请求时出现HTTPS不安全请求警告，curl测试API格式正常。

1. 检查QA链传递的完整Prompt格式 #

单独测试时使用的是格式合规的[INST]...[/INST]模板，但RetrievalQA拼接上下文后生成的Prompt可能存在格式问题（比如多余换行、标签未闭合等）。在_call方法中新增打印，确认传递给API的完整Prompt：

def _call(
    self,
    prompt: str,
    stop: Optional[List[str]] = None,
    run_manager: Optional[CallbackManagerForLLMRun] = None,
    **kwargs: Any,
) -> str:
    print("Full Prompt Sent to API:", prompt)  # 新增打印
    if stop is not None:
        raise ValueError("stop kwargs are not permitted.")
    # 后续代码不变

根据打印结果验证Prompt是否符合自研API的格式要求。

2. 捕获API返回的完整响应信息 #

当前代码仅通过response.raise_for_status()捕获HTTP状态码错误，但API可能返回200状态码却包含业务层错误。在_call方法中打印完整响应：

response = requests.post(self.llm_url, json=payload, headers=headers, verify=False)
response.raise_for_status()
full_response = response.json()
print("Full API Response:", full_response)  # 新增打印
return full_response['generated_text']

查看响应中是否包含error等字段，定位具体业务报错原因。

3. 保持参数传递一致性 #

对比curl测试和QA链调用的参数：

• 单独测试时max_new_tokens设为100，确保QA链调用时参数一致；
• 将自定义LLM类中的硬编码参数改为可配置项，避免参数不一致：

class LlamaLLM(LLM):
    llm_url: str = 'https:/myhost/llama/api'
    max_new_tokens: int = 100
    token: str = "abcdfejkwehr"

    class Config:
        extra = Extra.forbid

    # 其他方法不变
    def _call(...):
        payload = {
            "inputs": prompt,
            "parameters": {"max_new_tokens": self.max_new_tokens},
            "token": self.token
        }

4. 处理HTTPS警告（可选） #

虽然警告不影响功能，但可通过以下方式消除：

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

或添加API的CA证书路径：

response = requests.post(self.llm_url, json=payload, headers=headers, verify="/path/to/ca.crt")

内容的提问来源于stack exchange，提问作者jindo