使用llama_index查询本地PDF时遭遇Prompt类型错误求助

问题根源 #

你直接将LangChain的GPT4All实例传给了LlamaIndex的ServiceContext的llm_predictor参数，但LlamaIndex要求该参数必须是其自身的LLMPredictor实例，而非LangChain的LLM对象。这导致内部调用时，LlamaIndex传递的Prompt对象不符合LangChain LLM的输入要求（期望字符串），从而触发错误。

修复步骤 #

1. 导入LLMPredictor：从llama_index中导入LLMPredictor类
2. 创建LLMPredictor实例：用你的GPT4All对象初始化LLMPredictor
3. 传入ServiceContext：将LLMPredictor实例传给service_context的llm_predictor参数

修复后的完整代码 #

from llama_index import VectorStoreIndex, SimpleDirectoryReader
from InstructorEmbedding import INSTRUCTOR
from llama_index import PromptHelper, ServiceContext, LLMPredictor  # 新增导入
from llama_index import LangchainEmbedding
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.llms import GPT4All
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

documents = SimpleDirectoryReader(r'C:\Users\avish.wagde\Documents\work_avish\LLM_trials\instructor_large').load_data()

model_id = 'hkunlp/instructor-large'
model_path = "..\models\GPT4All-13B-snoozy.ggmlv3.q4_0.bin"

callbacks = [StreamingStdOutCallbackHandler()]
llm = GPT4All(model=model_path, callbacks=callbacks, verbose=True)

# 创建LLMPredictor实例
llm_predictor = LLMPredictor(llm=llm)

embed_model = LangchainEmbedding(HuggingFaceEmbeddings(model_name=model_id))

max_input_size = 4096
num_output = 256
max_chunk_overlap = 0.2
prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)

# 传入LLMPredictor而非原始LLM对象
service_context = ServiceContext.from_defaults(
    chunk_size=1024,
    llm_predictor=llm_predictor,  # 这里用llm_predictor
    prompt_helper=prompt_helper,
    embed_model=embed_model
)

index = VectorStoreIndex.from_documents(documents, service_context=service_context)
query_engine = index.as_query_engine()

response = query_engine.query("What is apple's financial situation")
print(response)

额外注意事项 #

• 检查LlamaIndex和LangChain的版本兼容性，建议使用较新的稳定版本避免API差异
• 确保model_path的路径正确，Windows下注意转义或使用原始字符串（比如r"..\models\..."）
• 若仍有问题，可尝试调整chunk_size和max_input_size，避免超过模型的上下文窗口限制

内容的提问来源于stack exchange，提问作者Avish Wagde