如何在基于llama-index的Python GPT-3应用中添加消息历史

我对用Python的LlamaIndex库训练GPT-3，以及通过标准API调用ChatGPT的经验不多。我知道在标准ChatGPT API里可以通过以下代码让模型参考消息历史作为上下文：

message_history=[]
completion = openai.ChatCompletion.create(model="gpt-3.5-turbo",messages=message_history)

我现在用LlamaIndex基于特定上下文训练GPT-3，但不知道怎么让模型同时考虑消息历史，以下是我当前的代码，求实现消息历史的方法：

def construct_index(directory_path):
    # set maximum input size
    max_input_size = 4096
    # set number of output tokens
    num_outputs = 2000
    # set maximum chunk overlap
    max_chunk_overlap = 20
    # set chunk size limit
    chunk_size_limit = 600 

    # define prompt helper
    prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)
    # define LLM
    llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="text-ada-001", max_tokens=num_outputs))
    # define context (dataset)
    documents = SimpleDirectoryReader(directory_path).load_data()
    # transform context to index format
    service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
    index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)
    # important: index are are like map, has latitutdes and logntitudes to indicate how each city (texts) are close to each other
    index.save_to_disk("index.json")
    return index

index = GPTSimpleVectorIndex.load_from_disk("index.json")
dbutils.widgets.text("user_input", "user: ")
response = index.query(dbutils.widgets.get("user_input"),response_mode='compact')
print("Response: ", response.response)

两种实现方案 #

方案1：手动拼接对话历史到查询 #

直接把过往的对话记录拼接到当前查询前，让模型在检索自定义文档时同时参考历史对话。修改后的代码如下：

# 初始化消息历史列表
message_history = []

index = GPTSimpleVectorIndex.load_from_disk("index.json")
dbutils.widgets.text("user_input", "user: ")

while True:
    user_input = dbutils.widgets.get("user_input")
    if user_input.lower() == "exit":
        break
    
    # 把历史对话拼接成字符串
    history_content = "\n".join([f"用户: {item['user']}\n助手: {item['assistant']}" for item in message_history])
    # 组合成完整查询
    full_query = f"{history_content}\n用户: {user_input}" if history_content else user_input
    
    # 执行查询
    response = index.query(full_query, response_mode='compact')
    print("Response: ", response.response)
    
    # 更新消息历史
    message_history.append({
        "user": user_input,
        "assistant": response.response
    })

方案2：用LlamaIndex原生ChatEngine（更推荐） #

LlamaIndex自带ChatEngine组件，能自动管理对话历史，同时无缝结合你的自定义索引上下文，代码更简洁：

from llama_index.chat_engine import SimpleChatEngine

def construct_index(directory_path):
    # 原索引构建代码保持不变
    max_input_size = 4096
    num_outputs = 2000
    max_chunk_overlap = 20
    chunk_size_limit = 600 

    prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)
    llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="text-ada-001", max_tokens=num_outputs))
    documents = SimpleDirectoryReader(directory_path).load_data()
    service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
    index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)
    index.save_to_disk("index.json")
    return index

index = GPTSimpleVectorIndex.load_from_disk("index.json")
# 创建ChatEngine实例，自动处理对话历史
chat_engine = SimpleChatEngine.from_defaults(index=index)

dbutils.widgets.text("user_input", "user: ")
while True:
    user_input = dbutils.widgets.get("user_input")
    if user_input.lower() == "exit":
        break
    response = chat_engine.chat(user_input)
    print("Response: ", response)

注意事项 #

• 注意模型的token上限，比如text-ada-001的上下文窗口有限，对话历史过长时要做截断，避免触发token超限错误。
• 如果想用聊天模型（比如gpt-3.5-turbo），把LLMPredictor里的model_name改成对应值即可，效果会更好：llm=OpenAI(temperature=0.5, model_name="gpt-3.5-turbo", max_tokens=num_outputs)

内容的提问来源于stack exchange，提问作者Lawrd_Das