使用Claude-v2时ContextualCompressionRetriever调用报错求助

问题原因 #

LLMChainFilter 默认的提示模板和输出解析器是为 OpenAI 模型设计的，要求模型输出严格的大写 YES 或 NO。但 Anthropic Claude-v2 的默认输出可能是小写 yes/no 或附带其他文本，导致 BooleanOutputParser 无法识别，触发错误。

解决方案 #

通过自定义提示模板和输出解析器，适配 Claude-v2 的输出逻辑：

方法1：自定义提示模板，强制输出大写 YES/NO #

明确要求 Claude 仅输出大写的 YES 或 NO，避免格式偏差：

from langchain.retrievers.document_compressors import LLMChainFilter
from langchain.prompts import PromptTemplate

# 适配Claude的提示模板，严格限定输出格式
claude_filter_prompt = PromptTemplate(
    input_variables=["question", "document"],
    template="""Given the question: {question}
And the document: {document}
Answer with ONLY 'YES' if the document has information relevant to the question, otherwise answer with ONLY 'NO'. Do not add any extra text, explanation, or punctuation."""
)

# 使用自定义提示创建压缩器
compressor = LLMChainFilter.from_llm(llm, prompt=claude_filter_prompt)
compression_retriever = ContextualCompressionRetriever(base_compressor=compressor, base_retriever=retriever)

方法2：自定义输出解析器，兼容大小写 #

如果 Claude 偶尔输出小写格式，修改解析器忽略大小写：

from langchain.retrievers.document_compressors import LLMChainFilter
from langchain.output_parsers import BooleanOutputParser

# 自定义兼容大小写的布尔解析器
class CaseInsensitiveBooleanOutputParser(BooleanOutputParser):
    def parse(self, text: str) -> bool:
        cleaned_text = text.strip().upper()
        if cleaned_text == "YES":
            return True
        elif cleaned_text == "NO":
            return False
        else:
            raise ValueError(f"Expected YES or NO, got {cleaned_text}")

# 创建解析器实例并替换默认配置
custom_parser = CaseInsensitiveBooleanOutputParser()
compressor = LLMChainFilter.from_llm(llm, output_parser=custom_parser)
compression_retriever = ContextualCompressionRetriever(base_compressor=compressor, base_retriever=retriever)

方法3：结合两种方式（推荐） #

同时使用自定义提示和兼容解析器，双重保障输出格式符合要求：

from langchain.retrievers.document_compressors import LLMChainFilter
from langchain.prompts import PromptTemplate
from langchain.output_parsers import BooleanOutputParser

class CaseInsensitiveBooleanOutputParser(BooleanOutputParser):
    def parse(self, text: str) -> bool:
        cleaned_text = text.strip().upper()
        if cleaned_text == "YES":
            return True
        elif cleaned_text == "NO":
            return False
        else:
            raise ValueError(f"Expected YES or NO, got {cleaned_text}")

claude_filter_prompt = PromptTemplate(
    input_variables=["question", "document"],
    template="""Given the question: {question}
And the document: {document}
Answer with ONLY 'YES' if the document has information relevant to the question, otherwise answer with ONLY 'NO'. Do not add any extra text, explanation, or punctuation."""
)

custom_parser = CaseInsensitiveBooleanOutputParser()
compressor = LLMChainFilter.from_llm(llm, prompt=claude_filter_prompt, output_parser=custom_parser)
compression_retriever = ContextualCompressionRetriever(base_compressor=compressor, base_retriever=retriever)

内容的提问来源于stack exchange，提问作者PlanetX