OpenAI+LangChain开发PDF聊天机器人遇模型弃用错误的解决求助

我是一名新手，正在用OpenAI和LangChain开发PDF聊天机器人，运行代码到chain.run(input_documents=docs, question=query)这一行时，碰到了InvalidRequestError错误，提示模型text-davinci-003已经被弃用。以下是我的原代码，需要替换成当前支持的模型：

from PyPDF2 import PdfReader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import ElasticVectorSearch, Pinecone, Weaviate, FAISS
from langchain.chains import RetrievalQA
import os
os.environ["OPENAI_API_KEY"] = "API KEY"
pdfs_folder = 'PDFs/'
pdf_files = [file for file in os.listdir(pdfs_folder) if file.endswith('.pdf')]
raw_text = ''
# Iterate through each PDF file
for pdf_file in pdf_files:
    # Construct the path to the PDF file
    pdf_path = os.path.join(pdfs_folder, pdf_file)
    # Read the PDF file
    with open(pdf_path, 'rb') as file:
        reader = PdfReader(file)
        
        # Extract raw text from pages
        for i, page in enumerate(reader.pages):
            text = page.extract_text()
            if text:
                raw_text += text
text_splitter = CharacterTextSplitter(        
    separator = "\n",
    chunk_size = 500,
    chunk_overlap  = 200,
    length_function = len,
)
chunks = text_splitter.split_text(raw_text)
embeddings = OpenAIEmbeddings()
docsearch = FAISS.from_texts(chunks, embeddings)
chain = load_qa_chain(OpenAI(), chain_type="stuff")
query = "What are the core values of JM Finance?"
docs = docsearch.similarity_search(query)
chain.run(input_documents=docs, question=query)

修改方案 #

text-davinci-003属于OpenAI旧版的Completion模型，现已被弃用，推荐使用GPT-3.5或GPT-4系列的聊天模型，具体修改步骤如下：

• 新增导入ChatOpenAI类（LangChain中用于调用OpenAI聊天模型的接口）
• 将原代码中的OpenAI()替换为ChatOpenAI(model="gpt-3.5-turbo")，也可以根据需要换成gpt-4或gpt-4-turbo等当前支持的模型

修改后的完整代码 #

from PyPDF2 import PdfReader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import ElasticVectorSearch, Pinecone, Weaviate, FAISS
from langchain.chains import load_qa_chain  # 补全原代码缺失的导入
from langchain.chat_models import ChatOpenAI  # 新增导入聊天模型类
import os
os.environ["OPENAI_API_KEY"] = "API KEY"
pdfs_folder = 'PDFs/'
pdf_files = [file for file in os.listdir(pdfs_folder) if file.endswith('.pdf')]
raw_text = ''
# 遍历所有PDF文件
for pdf_file in pdf_files:
    # 构建PDF文件路径
    pdf_path = os.path.join(pdfs_folder, pdf_file)
    # 读取PDF文件
    with open(pdf_path, 'rb') as file:
        reader = PdfReader(file)
        
        # 提取每页文本
        for i, page in enumerate(reader.pages):
            text = page.extract_text()
            if text:
                raw_text += text
text_splitter = CharacterTextSplitter(        
    separator = "\n",
    chunk_size = 500,
    chunk_overlap  = 200,
    length_function = len,
)
chunks = text_splitter.split_text(raw_text)
embeddings = OpenAIEmbeddings()
docsearch = FAISS.from_texts(chunks, embeddings)
# 替换为当前支持的聊天模型
chain = load_qa_chain(ChatOpenAI(model="gpt-3.5-turbo"), chain_type="stuff")
query = "JM Finance的核心价值观是什么？"
docs = docsearch.similarity_search(query)
# 新版LangChain推荐用invoke方法替代run，兼容性更好
response = chain.invoke({"input_documents": docs, "question": query})
print(response['output_text'])

额外说明 #

• 确保LangChain版本为最新，可通过pip install --upgrade langchain更新
• 使用GPT-4系列模型需要你的OpenAI账号具备调用权限
• invoke方法是LangChain新版推荐的调用方式，比旧版run更稳定

内容的提问来源于stack exchange，提问作者Andy