加载meta-llama/Llama-2-7b-chat-hf模型时遇HTTPS连接超时求助

问题描述 #

运行以下代码加载模型时，下载分片到29%触发连接超时错误：

from getpass import getpass
import os
HUGGINGFACE_API_TOKEN = getpass()
os.environ[HUGGINGFACE_API_TOKEN] = HUGGINGFACE_API_TOKEN
from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-chat-hf")

错误信息：

TimeoutError Traceback (most recent call last)
File ~/.local/lib/python3.10/site-packages/urllib3/response.py:438, in HTTPResponse._error_catcher(self)
437 try:
--> 438 yield
440 except SocketTimeout:
441 # FIXME: Ideally we'd like to include the url in the ReadTimeoutError but
442 # there is yet no clean way to get at it from this context.

File ~/.local/lib/python3.10/site-packages/requests/models.py:822, in Response.iter_content..generate()
820 raise ContentDecodingError(e)
821 except ReadTimeoutError as e:
--> 822 raise ConnectionError(e)
823 except SSLError as e:
824 raise RequestsSSLError(e)

ConnectionError: HTTPSConnectionPool(host='cdn-lfs.huggingface.co', port=443): Read timed out.

解决方法 #

1. 修正代码错误并增加下载稳定性参数 #

你代码中的环境变量设置有误，正确的变量名是HUGGINGFACE_HUB_TOKEN。同时在加载模型时添加超时、重试和断点续传参数，提升下载稳定性：

from getpass import getpass
import os
from transformers import AutoTokenizer, AutoModelForCausalLM

# 正确配置认证token
HUGGINGFACE_API_TOKEN = getpass()
os.environ["HUGGINGFACE_HUB_TOKEN"] = HUGGINGFACE_API_TOKEN

# 加载tokenizer，增加超时与重试
tokenizer = AutoTokenizer.from_pretrained(
    "meta-llama/Llama-2-7b-chat-hf",
    timeout=300,  # 设置5分钟超时
    retry=3,      # 失败后重试3次
    use_auth_token=True
)

# 加载模型，开启断点续传
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-2-7b-chat-hf",
    timeout=300,
    retry=3,
    use_auth_token=True,
    resume_download=True  # 已下载部分不会重复下载
)

2. 手动下载模型到本地加载 #

如果自动下载持续超时，可手动下载模型所有文件到本地目录（比如./llama-2-7b-chat），再通过本地路径加载：

tokenizer = AutoTokenizer.from_pretrained("./llama-2-7b-chat")
model = AutoModelForCausalLM.from_pretrained("./llama-2-7b-chat")

3. 配置网络代理（网络受限场景） #

若无法直接访问Hugging Face CDN，可设置代理后再执行下载：

import os
os.environ["HTTP_PROXY"] = "http://你的代理地址:端口"
os.environ["HTTPS_PROXY"] = "http://你的代理地址:端口"

# 后续执行模型加载代码

4. 切换稳定网络 #

尝试切换到有线网络，或避开网络高峰时段下载，降低连接中断概率。

内容的提问来源于stack exchange，提问作者Rishabh Gupta