在Colab中使用Hugging Face加载Video-LLaVA遇文件缺失错误求助

问题场景 #

在Colab中使用Hugging Face加载LanguageBind/Video-LLaVA-7B模型时，触发404错误，提示仓库中缺少preprocessor_config.json文件。

错误日志 #

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py in hf_raise_for_status(response, endpoint_name)
    285     try:
--> 286         response.raise_for_status()
    287     except HTTPError as e:

15 frames
HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/LanguageBind/Video-LLaVA-7B/resolve/main/preprocessor_config.json

The above exception was the direct cause of the following exception:

EntryNotFoundError                        Traceback (most recent call last)
EntryNotFoundError: 404 Client Error. (Request ID: Root=1-65d6e313-109341163745f316481be0ac;97f03b13-3bac-447e-b9df-a109f451bc7c)

Entry Not Found for url: https://huggingface.co/LanguageBind/Video-LLaVA-7B/resolve/main/preprocessor_config.json.

The above exception was the direct cause of the following exception:

OSError                                   Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, **deprecated_kwargs)
    434         if revision is None:
    435             revision = "main"
--> 436         raise EnvironmentError(
    437             f"{path_or_repo_id} does not appear to have a file named {full_filename}. Checkout "
    438             f"'https://huggingface.co/{path_or_repo_id}/{revision}' for available files."

OSError: LanguageBind/Video-LLaVA-7B does not appear to have a file named preprocessor_config.json. Checkout 'https://huggingface.co/LanguageBind/Video-LLaVA-7B/main' for available files.

运行代码 #

from transformers import AutoProcessor, AutoModelForCausalLM

processor = AutoProcessor.from_pretrained("LanguageBind/Video-LLaVA-7B")
model = AutoModelForCausalLM.from_pretrained("LanguageBind/Video-LLaVA-7B")

解决方法 #

方法一：使用官方自定义加载逻辑（推荐） #

Video-LLaVA属于LanguageBind系列模型，官方未完全适配transformers的AutoProcessor/AutoModelForCausalLM自动加载逻辑，需通过克隆仓库使用自定义代码加载：

# 克隆LanguageBind仓库
!git clone https://github.com/PKU-YuanGroup/LanguageBind.git
%cd LanguageBind/Video-LLaVA
# 安装依赖
!pip install -r requirements.txt
# 加载模型和处理器
from video_llava.constants import IMAGE_TOKEN_INDEX
from video_llava.conversation import conv_templates, SeparatorStyle
from video_llava.model.builder import load_pretrained_model
from video_llava.utils import disable_torch_init
from video_llava.mm_utils import process_images, tokenizer_image_token, get_model_name_from_path

# 初始化环境
disable_torch_init()
model_path = "LanguageBind/Video-LLaVA-7B"
model_name = get_model_name_from_path(model_path)
# 加载模型、分词器和处理器
tokenizer, model, processor, context_len = load_pretrained_model(model_path, None, model_name)

方法二：手动创建preprocessor_config.json（不推荐） #

若坚持使用transformers的Auto类，可手动创建preprocessor_config.json文件，放入模型缓存目录或仓库中，参考同系列模型配置示例：

{
  "image_processor": {
    "crop_size": {
      "height": 224,
      "width": 224
    },
    "do_center_crop": true,
    "do_normalize": true,
    "do_resize": true,
    "image_mean": [0.48145466, 0.4578275, 0.40821073],
    "image_std": [0.26862954, 0.26130258, 0.27577711],
    "resample": 3,
    "size": {
      "shortest_edge": 224
    }
  },
  "tokenizer": {
    "model_max_length": 2048,
    "padding_side": "right",
    "truncation_side": "right",
    "tokenizer_class": "LlamaTokenizer",
    "use_fast": false
  },
  "video_processor": {
    "do_normalize": true,
    "do_resize": true,
    "image_mean": [0.48145466, 0.4578275, 0.40821073],
    "image_std": [0.26862954, 0.26130258, 0.27577711],
    "resample": 3,
    "size": {
      "shortest_edge": 224
    },
    "num_frames": 8
  }
}

注意：此方法可能因参数与模型实际要求不匹配导致后续推理出错，仅作为临时方案。

内容的提问来源于stack exchange，提问作者Kamakshi Ramamurthy