Pinecone similarity_search返回空结果问题排查求助

我正在学习使用Pinecone，但执行similarity_search操作时始终返回空结果，尽管索引已成功创建并填充。以下是完整代码：

import os
import time
from dotenv import load_dotenv, find_dotenv
from langchain_community.document_loaders import PDFMinerLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from pinecone import Pinecone
from langchain_pinecone import PineconeVectorStore
from pinecone import ServerlessSpec
from langchain.chains.question_answering import load_qa_chain
from langchain.chat_models import ChatOpenAI

load_dotenv(find_dotenv(), override=True)

pdf_file_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'pdf_files', 'document_1.pdf')
loader = PDFMinerLoader(pdf_file_path)
document = loader.load()

embedding = OpenAIEmbeddings()

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size = 512,
    chunk_overlap = 100,
    separators=[""])
chunks = text_splitter.split_documents(document)

pc = Pinecone()

index_name = 'tenders-index'

for i in pc.list_indexes().names():
    pc.delete_index(i)
    print('Index deleted')

if index_name not in pc.list_indexes().names():
    print(f'Creating index {index_name}')
    pc.create_index(
        name=index_name,
        dimension=1536,
        metric='cosine',
        spec=ServerlessSpec(
            cloud="aws",
            region="us-east-1"
        )
    )
    print('Index created!')
else:
    print(f'Index {index_name} already exists!')

while not pc.describe_index(index_name).status['ready']:
    time.sleep(1)

vector_store = PineconeVectorStore.from_documents(
    documents= chunks,
    embedding=embedding,
    index_name=index_name
)

query = 'The goal of this document is ...'

vstore = PineconeVectorStore.from_existing_index(index_name=index_name, embedding=embedding)

llm = ChatOpenAI(model='gpt-3.5-turbo')

docs = vstore.similarity_search(query=query, k=3)

chain = load_qa_chain(
    llm = llm,
    chain_type = 'stuff'
)

response = chain.run(input_documents = docs , question = query)

print(response)

print('Done!')

已完成的排查：

• 索引已存在
• Pinecone索引与OpenAIEmbeddings维度一致
• VectorStore已生成

可能的解决方向 #

1. 验证PDF加载有效性
   PDFMinerLoader无法读取加密PDF或图片型PDF（扫描件），添加调试代码确认文档是否加载成功：

   print(f"Loaded document count: {len(document)}")
   if document:
       print(f"First document content preview: {document[0].page_content[:200]}...")
   

2. 检查文本分割结果
   当前separators=[""]可能导致分割逻辑异常，无法生成有效chunk。建议调整为默认分隔符组合：

   text_splitter = RecursiveCharacterTextSplitter(
       chunk_size=512,
       chunk_overlap=100,
       separators=["\n\n", "\n", " ", ""]
   )
   

   同时添加打印确认chunk数量：

   print(f"Generated chunk count: {len(chunks)}")
   if chunks:
       print(f"First chunk content preview: {chunks[0].page_content[:200]}...")
   

3. 确认向量入库成功
   入库后打印索引统计信息，验证是否有向量存入：

   stats = vector_store._index.describe_index_stats()
   print(f"Index statistics: {stats}")
   

   也可直接登录Pinecone控制台查看索引的向量数量。

4. 优化查询语句
   如果查询与文档内容关联性极低，cosine相似度可能低于阈值导致返回空。尝试使用similarity_search_with_score查看匹配分数：

   docs_with_score = vstore.similarity_search_with_score(query=query, k=3)
   print(f"Search results with score: {docs_with_score}")
   

   同时调整查询语句，使其更贴合文档内容。

5. 确认环境变量加载
   检查OPENAI_API_KEY和PINECONE_API_KEY是否通过.env文件正确加载，可添加打印验证：

   print(f"OpenAI API Key loaded: {os.getenv('OPENAI_API_KEY') is not None}")
   print(f"Pinecone API Key loaded: {os.getenv('PINECONE_API_KEY') is not None}")
   

内容的提问来源于stack exchange，提问作者Felipe Acosta