使用pickle序列化FAISS对象时遇无法序列化_thread.RLock错误求助

在使用LangChain构建基于OpenAI嵌入的FAISS向量索引时，尝试用pickle.dump()保存索引对象时触发以下错误：

TypeError: cannot pickle '_thread.RLock' object

原因是FAISS对象内部包含线程锁这类无法被Pickle序列化的组件，不能直接用Pickle保存。

解决方案 #

直接使用FAISS内置的save_local()方法保存索引，加载时用load_local()方法，这是LangChain官方推荐的序列化方式，无需依赖Pickle。

替换保存代码 #

将原代码中的Pickle保存逻辑：

# Storing vector index create in local
file_path="vector_index.pkl"
with open(file_path, "wb") as f:
    pickle.dump(vectorindex_openai, f)

替换为FAISS自带的保存方法：

# 使用FAISS内置方法保存向量索引
vectorindex_openai.save_local("faiss_index")

加载保存的索引（可选） #

后续需要复用索引时，使用以下代码加载：

from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings

embeddings = OpenAIEmbeddings()
vectorindex_openai = FAISS.load_local("faiss_index", embeddings)

修改后的完整代码 #

# -*- coding: utf-8 -*-
"""Langchain_LLM.ipynb

Automatically generated by Colaboratory.

Original file is located at
    https://colab.research.google.com/drive/1DWToK3XFOM0v5bl7-LwT0GBfKyYVulnb
"""

!pip install python-magic langchain unstructured streamlit openai tiktoken faiss-gpu

import os
import streamlit as st
import time
from langchain import OpenAI
from langchain.chains import RetrievalQAWithSourcesChain
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.document_loaders import UnstructuredURLLoader
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS

os.environ['OPENAI_API_KEY'] = "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"

llm = OpenAI(temperature = 0.9, max_tokens=500)

loader = UnstructuredURLLoader(
    urls = [
        "https://www.moneycontrol.com/news/business/banks/hdfc-bank-re-appoints-sanmoy-chakrabarti-as-chief-risk-officer-11259771.html",
        "https://www.moneycontrol.com/news/business/markets/market-corrects-post-rbi-ups-inflation-forecast-icrr-bet-on-these-top-10-rate-sensitive-stocks-ideas-11142611.html"
    ]
)
data = loader.load()

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size = 1000,
    chunk_overlap  = 200,
)
docs = text_splitter.split_documents(data)

embeddings = OpenAIEmbeddings()
vectorindex_openai = FAISS.from_documents(docs, embeddings)

# 替换Pickle保存为FAISS内置方法
vectorindex_openai.save_local("faiss_index")

内容的提问来源于stack exchange，提问作者Sharath Kumar Reddy