开发PDF聊天机器人时遇LlamaCPP导入及模块缺失错误求助

问题 #

开发PDF聊天机器人时，运行代码遇到两类错误：

1. ImportError: cannot import name 'LlamaCPP' from 'llama_index.llms' (unknown location)
2. ModuleNotFoundError: No module named 'llama_index.llms.llama_utils'

运行的代码如下：

import torch

#from llama_index.llms import LlamaCPP
from llama_index.llms.llama_utils import messages_to_prompt, completion_to_prompt
llm = LlamaCPP(
    
    model_url='https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q4_K_M.gguf',
    # optionally, you can set the path to a pre-downloaded model instead of model_url
    model_path=None,
    temperature=0.1,
    max_new_tokens=256,
    # llama2 has a context window of 4096 tokens, but we set it lower to allow for some wiggle room
    context_window=3900,
    # kwargs to pass to __call__()
    generate_kwargs={},
    # kwargs to pass to __init__()
    # set to at least 1 to use GPU
    model_kwargs={"n_gpu_layers": -1},
    # transform inputs into Llama2 format
    messages_to_prompt=messages_to_prompt,
    completion_to_prompt=completion_to_prompt,
    verbose=True,
)

解决方案 #

• 修正导入路径：llama-index版本更新后，模块结构调整，LlamaCPP及对应的prompt工具函数已移至llama_index.llms.llama_cpp路径下，修改导入代码如下：

  import torch
  from llama_index.llms.llama_cpp import LlamaCPP
  from llama_index.llms.llama_cpp import messages_to_prompt, completion_to_prompt
  

• 重装依赖确保版本匹配：若调整导入后仍报错，先卸载旧版本依赖，安装最新稳定版：

  pip uninstall -y llama-index llama-cpp-python
  pip install llama-index llama-cpp-python
  

• 适配参数名（可选）：部分新版本中model_url参数已更名为download_url，若遇到参数未定义错误，调整初始化代码中的对应参数：

  llm = LlamaCPP(
      download_url='https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q4_K_M.gguf',
      model_path=None,
      temperature=0.1,
      max_new_tokens=256,
      context_window=3900,
      generate_kwargs={},
      model_kwargs={"n_gpu_layers": -1},
      messages_to_prompt=messages_to_prompt,
      completion_to_prompt=completion_to_prompt,
      verbose=True,
  )
  

内容的提问来源于stack exchange，提问作者shubham joshi2014