使用Chroma.from_documents时遇AttributeError: hnswlib.Index无file_handle_count属性

环境信息 #

• 设备：Macbook M2
• Python版本：3.9.18
• ChromaDB版本：0.4.18

问题说明 #

执行PDF文档嵌入并加载到ChromaDB向量存储的代码时，触发AttributeError: type object 'hnswlib.Index' has no attribute 'file_handle_count'错误。

运行代码 #

# test with one doc
from langchain.document_loaders import PyPDFLoader

# load data
loader = PyPDFLoader(".XXX.pdf")

# get list of documents
pages = loader.load() 

# split
text_splitter = CharacterTextSplitter(
    separator="\n",
    chunk_size=450,
    chunk_overlap=50,
    length_function=len 
    )

#print(page)
pdf_splits = text_splitter.split_documents(pages) # list of documents
print(pdf_splits[:2])
print(len(pages), len(pdf_splits))

# create a list of texts
text_list = []
for doc in pdf_splits:
    text_list.append(doc.page_content)

# embedding
#rm -rf ../chatbot_mvp/vectordb/PMS_research # removes inital store 
embedding = OpenAIEmbeddings(openai_api_key=XXX)
embedding.embed_documents(text_list) 

# vectior store # HELP ERROR
persist_directory = '../chatbot_mvp/vectordb/XXX/'
vectordb = Chroma.from_documents(
    documents=pdf_splits,
    embedding=embedding,
    persist_directory=persist_directory)

错误日志 #

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/Users/karinwiberg/Documents/chatbot_development/chatbot_mvp/chatbot_mvp.ipynb Cell 16 line 3
     33 # vectior store # HELP ERROR
     34 persist_directory = '../chatbot_mvp/vectordb/PMS_research/'
---> 35 vectordb = Chroma.from_documents(
     36     documents=pdf_splits,
     37     embedding=embedding,
     38     persist_directory=persist_directory)

File ~/opt/anaconda3/envs/femai-alicia-prototype-01/lib/python3.9/site-packages/langchain/vectorstores/chroma.py:771, in Chroma.from_documents(cls, documents, embedding, ids, collection_name, persist_directory, client_settings, client, collection_metadata, **kwargs)
    769 texts = [doc.page_content for doc in documents]
    770 metadatas = [doc.metadata for doc in documents]
---> 771 return cls.from_texts(
    772     texts=texts,
    773     embedding=embedding,
    774     metadatas=metadatas,
    775     ids=ids,
    776     collection_name=collection_name,
    777     persist_directory=persist_directory,
    778     client_settings=client_settings,
    779     client=client,
    780     collection_metadata=collection_metadata,
    781     **kwargs,
    782 )
...
---> 445     hnswlib_count = hnswlib.Index.file_handle_count
    446     hnswlib_count = cast(int, hnswlib_count)
    447     # One extra for the metadata file

AttributeError: type object 'hnswlib.Index' has no attribute 'file_handle_count'

已尝试的解决方案 #

• 安装了chroma-hnswlib-0.7.1、chromadb-0.4.3、fastapi-0.99.1、pydantic-1.10.13，但问题未解决
• 看到其他用户建议降级ChromaDB至0.4.3，但该版本在官方文档中未找到发布记录

内容的提问来源于stack exchange，提问作者karwi