在Google Colab搭建Llama3 RAG时遭遇ConnectionRefusedError

问题 #

在Google Colab上搭建基于Llama3的RAG系统，使用以下代码：

#### INDEXING ####

# Load Documents
loader = WebBaseLoader(
    web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
    bs_kwargs=dict(
        parse_only=bs4.SoupStrainer(
            class_("post-content", "post-title", "post-header")
        )
    ),
)
docs = loader.load()

# Split
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
splits = text_splitter.split_documents(docs)

# Embed
vectorstore = Chroma.from_documents(
    documents=splits, 
    embedding=OllamaEmbeddings(model=local_llm)
)

retriever = vectorstore.as_retriever()

执行到Chroma.from_documents步骤时，出现连接拒绝错误：

ConnectionError: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/embeddings (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7c949f725420>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)

<ipython-input-9-2332286f4c58> in <cell line: 29>()
     27 
     28 # Embed
---> 29 vectorstore = Chroma.from_documents(
     30     documents=splits,
     31     embedding=OllamaEmbeddings(model=local_llm)

/usr/local/lib/python3.10/dist-packages/langchain_community/vectorstores/chroma.py in from_documents(cls, documents, embedding, ids, collection_name, persist_directory, client_settings, client, collection_metadata, **kwargs)
    788         texts = [doc.page_content for doc in documents]
    789         metadatas = [doc.metadata for doc in documents]
---> 790         return cls.from_texts(
    791             texts=texts,
    792             embedding=embedding,

/usr/local/lib/python3.10/dist-packages/langchain_community/vectorstores/chroma.py in from_texts(cls, texts, embedding, metadatas, ids, collection_name, persist_directory, client_settings, client, collection_metadata, **kwargs)
    746                 documents=texts,
    747             ):
---> 748                 chroma_collection.add_texts(
    749                     texts=batch[3] if batch[3] else [],
    750                     metadatas=batch[2] if batch[2] else None,

/usr/local/lib/python3.10/dist-packages/langchain_community/vectorstores/chroma.py in add_texts(self, texts, metadatas, ids, **kwargs)
    274         texts = list(texts)
    275         if self._embedding_function is not None:
---> 276             embeddings = self._embedding_function.embed_documents(texts)
    277         if metadatas:
    278             # fill metadatas with empty dicts if somebody

/usr/local/lib/python3.10/dist-packages/langchain_community/embeddings/ollama.py in embed_documents(self, texts)
    209         """
    210         instruction_pairs = [f"{self.embed_instruction}{text}" for text in texts]
---> 211         embeddings = self._embed(instruction_pairs)
    212         return embeddings
    213 
/usr/local/lib/python3.10/dist-packages/langchain_community/embeddings/ollama.py in _embed(self, input)
    197         else:
    198             iter_ = input
---> 199         return [self._process_emb_response(prompt) for prompt in iter_]
    200 
    201     def embed_documents(self, texts: List[str]) -> List[List[float]]:

/usr/local/lib/python3.10/dist-packages/langchain_community/embeddings/ollama.py in <listcomp>(.0)
    197         else:
    198             iter_ = input
---> 199         return [self._process_emb_response(prompt) for prompt in iter_]
    200 
    201     def embed_documents(self, texts: List[str]) -> List[List[float]]:

/usr/local/lib/python3.10/dist-packages/langchain_community/embeddings/ollama.py in _process_emb_response(self, input)
    168             )
    169         except requests.exceptions.RequestException as e:
---> 170             raise ValueError(f"Error raised by inference endpoint: {e}")
    171 
    172         if res.status_code != 200:

ValueError: Error raised by inference endpoint: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/embeddings (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7c949f725420>: Failed to establish a new connection: [Errno 111] Connection refused'))

已通过! pip install langchain和! pip install chromadb安装依赖，需要解决连接拒绝问题。

解决方案 #

错误核心是Google Colab环境未安装并启动Ollama服务，OllamaEmbeddings需要连接本地11434端口的Ollama API，按以下步骤操作：

• 安装Ollama：

  !curl https://ollama.ai/install.sh | sh
  

• 后台启动Ollama服务：

  !nohup ollama serve &
  

• 等待10-15秒让服务启动，拉取Llama3模型：

  !ollama pull llama3
  

• 验证服务状态：

  !curl http://localhost:11434/api/tags
  

  返回包含llama3的模型列表即为正常。
• 修正代码语法错误：原代码中class_("post-content", "post-title", "post-header")需改为class_=["post-content", "post-title", "post-header"]，否则会导致文档加载失败。
• 确保代码中模型名匹配：

  local_llm = "llama3"
  

内容的提问来源于stack exchange，提问作者Barry