创建Qdrant向量库遇维度不匹配错误,如何正确配置VectorParams?
问题解决:Qdrant向量参数配置不匹配
错误原因
你遇到的ValueError: could not broadcast input array from shape (1536,) into shape (2000,)是因为Qdrant集合配置的向量维度(2000)和OpenAI嵌入模型输出的向量维度(1536)不匹配。
OpenAI的text-embedding-ada-002模型生成的向量固定为1536维,这是模型的固有属性,不需要计算,直接使用官方指定的维度值即可。
正确配置方式
方式1:硬编码正确维度
直接将VectorParams的size参数改为1536,距离度量可根据需求选择(Distance.EUCLID或文本场景更常用的Distance.COSINE):
qdrantClient.create_collection( collection_name=collection_name, vectors_config=VectorParams(size=1536, distance=Distance.EUCLID), )
方式2:动态获取向量维度(推荐)
如果后续可能更换嵌入模型,为避免手动修改维度,可通过嵌入模型的embed_query方法动态获取向量维度,提升代码灵活性:
# 获取模型输出的向量维度 test_embedding = embeddings.embed_query("test") vector_dim = len(test_embedding) qdrantClient.create_collection( collection_name=collection_name, vectors_config=VectorParams(size=vector_dim, distance=Distance.EUCLID), )
修改后的完整代码
import os from typing import List from langchain.docstore.document import Document from langchain.embeddings import OpenAIEmbeddings from langchain.text_splitter import RecursiveCharacterTextSplitter from langchain.vectorstores import Qdrant, VectorStore from qdrant_client import QdrantClient from qdrant_client.models import Distance, VectorParams def load_documents(documents: List[Document]) -> VectorStore: """Create a vectorstore from documents.""" collection_name = "my_collection" vectorstore_path = "data/vectorstore/qdrant" embeddings = OpenAIEmbeddings( model="text-embedding-ada-002", openai_api_key=os.getenv("OPENAI_API_KEY"), ) qdrantClient = QdrantClient(path=vectorstore_path, prefer_grpc=True) # 动态获取向量维度 test_embedding = embeddings.embed_query("test") vector_dim = len(test_embedding) qdrantClient.create_collection( collection_name=collection_name, vectors_config=VectorParams(size=vector_dim, distance=Distance.EUCLID), ) vectorstore = Qdrant( client=qdrantClient, collection_name=collection_name, embeddings=embeddings, ) text_splitter = RecursiveCharacterTextSplitter( chunk_size=1000, chunk_overlap=200, ) sub_docs = text_splitter.split_documents(documents) vectorstore.add_documents(sub_docs) return vectorstore
内容的提问来源于stack exchange,提问作者Evan P
相关产品推荐
相关产品推荐

