Langchain构建LLM问答系统上下文前置打印问题求助

问题描述 #

基于Langchain框架结合GPT4All、FAISS、HuggingFace Embeddings搭建的本地问答系统，存在输出前强制打印上下文的问题，既造成内容冗余，又因上下文打印耗时15-20秒大幅降低响应速度。调整后要么LLM无法调用上下文生成有效回答，要么上下文仍会被打印。

问题根源 #

1. 流式回调打印所有内容：使用的StreamingStdOutCallbackHandler会将LLM生成的所有内容（包括Prompt模板中的Context、Question部分）实时输出到控制台。
2. Prompt模板设计导致冗余输出：原模板要求LLM输出完整的Context+Question+Answer结构，使得LLM会把上下文和问题重新打印一遍。
3. 上下文未精简：未利用筛选后的best_sentences，仍传入全部匹配文档内容，增加LLM处理时间和输出冗余。

解决方案 #

1. 移除流式打印回调，关闭verbose模式 #

将默认的流式输出回调替换为静默处理，同时关闭LLM的verbose日志，避免不必要的打印：

# 替换原LLM初始化代码
llm = GPT4All(model=gpt4all_path, verbose=False)

如果需要保留回调逻辑但不打印，可自定义空回调：

from langchain.callbacks.base import BaseCallbackHandler

class SilentCallbackHandler(BaseCallbackHandler):
    def on_llm_new_token(self, token: str, **kwargs) -> None:
        pass

callback_manager = CallbackManager([SilentCallbackHandler()])
llm = GPT4All(model=gpt4all_path, callback_manager=callback_manager, verbose=False)

2. 优化Prompt模板，仅要求生成答案 #

修改模板结构，让LLM只聚焦于生成答案部分，避免输出上下文和问题：

# 替换原模板定义
template = """
基于以下上下文回答问题：
{context}

问题：{question}

答案：
"""

3. 精简LLM调用逻辑，简化结果处理 #

移除不必要的参数，直接使用LLM生成的内容作为答案，无需截取：

# 修改answer_question函数中的LLM调用部分
def answer_question(question):
    # ... 保留原有匹配、筛选逻辑 ...
    
    # 用筛选后的best_sentences作为上下文（可选，大幅提升速度）
    context = "\n".join(best_sentences)
    
    prompt_template = PromptTemplate(template=template, input_variables=["context","question"])
    llm_chain = LLMChain(prompt=prompt_template, llm=llm)
    
    # 直接生成答案，无需传入answer参数
    generated_text = llm_chain.run(context=context, question=question, max_tokens=512, temperature=0.0, top_p=0.05)
    
    # 直接返回生成的答案内容
    return generated_text.strip()

4. 完整修改后的代码示例 #

import os
import nltk
from langchain import PromptTemplate, LLMChain
from langchain.llms import GPT4All
from langchain.document_loaders import UnstructuredPDFLoader, PyPDFLoader, DirectoryLoader, TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.indexes import VectorstoreIndexCreator
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores.faiss import FAISS
from transformers import pipeline
import faiss

# Define the paths
gpt4all_path = './models/gpt4all-converted.bin'

# Create the embeddings and llm objects
embeddings = HuggingFaceEmbeddings(model_name='sentence-transformers/all-MiniLM-L6-v2')
# 移除流式回调，关闭verbose
llm = GPT4All(model=gpt4all_path, verbose=False)

# Load the local index
index = FAISS.load_local("my_faiss_index", embeddings)

# Initialize the question-answering model
qa_model = pipeline("question-answering", model="distilbert-base-cased-distilled-squad", tokenizer="distilbert-base-cased")

# 优化后的Prompt模板
template = """
基于以下上下文回答问题：
{context}

问题：{question}

答案：
"""

# Define the similarity search function
def similarity_search(query, index, k=3):
    try:
        matched_docs = index.similarity_search(query, k=k)
        return matched_docs
    except Exception as e:
        print("An error occurred during similarity search: ", e)
        return []

# Split the documents into sentences
def split_into_sentences(document):
    return nltk.sent_tokenize(document)

# Select the best sentences based on the question
def select_best_sentences(question, sentences):
    results = []
    for sentence in sentences:
        answer = qa_model(question=question, context=sentence)
        if answer['score'] > 0.8:  # You can tune this threshold based on your requirements
            results.append(sentence)
    return results

def answer_question(question):
    # Get the most similar documents
    matched_docs = similarity_search(question, index)

    # Convert the matched documents into a list of sentences
    sentences = []
    for doc in matched_docs:
        sentences.extend(split_into_sentences(doc.page_content))

    # Select the best sentences
    best_sentences = select_best_sentences(question, sentences)

    # 使用筛选后的最佳句子作为上下文，减少输入长度
    context = "\n".join(best_sentences)

    # Create the prompt template
    prompt_template = PromptTemplate(template=template, input_variables=["context","question"])

    # Initialize the LLMChain
    llm_chain = LLMChain(prompt=prompt_template, llm=llm)

    # Generate the answer
    generated_text = llm_chain.run(context=context, question=question, max_tokens=512, temperature=0.0, top_p=0.05)
    
    # 直接返回清理后的答案
    return generated_text.strip()

# Main loop for continuous question-answering
while True:
    # Get the user's question
    question = input("Chatbot: ")

    # Check if the user wants to exit
    if question.lower() == "exit":
        break

    # Generate the answer
    answer = answer_question(question)

    # Print the answer
    print("Answer:", answer)

内容的提问来源于stack exchange，提问作者Abhishek Rai