构建含Hugging Face模型的Docker镜像时遇LocalEntryNotFoundError

问题场景 #

尝试通过Docker镜像创建AWS Lambda函数，在构建阶段执行dependency.py下载Hugging Face模型（hkunlp/instructor-large和SamLowe/roberta-base-go_emotions），但构建时出现缓存未找到的网络相关错误。本地运行该脚本正常，且镜像内可通过pip正常安装依赖。

相关代码 #

dependency.py

import logging
logging.basicConfig(level=logging.DEBUG)
from langchain.embeddings import HuggingFaceInstructEmbeddings

from transformers import AutoModelForSequenceClassification, AutoTokenizer

instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-large",
                                                      model_kwargs={"device": "cpu"},encode_kwargs={"batch_size": 1})

# Replace 'model_id' with your specific model identifier
model_id = "SamLowe/roberta-base-go_emotions"

# Download the model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained(model_id,force_download=True)
tokenizer = AutoTokenizer.from_pretrained(model_id,force_download=True)

# Save the model and tokenizer to a directory (e.g., "./models/")
model.save_pretrained("./model")
tokenizer.save_pretrained("./tokenizer")

Dockerfile

# lambda base image for Docker from AWS
FROM public.ecr.aws/lambda/python:latest

# Update the package list and install development tools for C++ support
RUN yum update -y && \
    yum install deltarpm -y && \
    yum groupinstall "Development Tools" -y 

# Export CXXFLAGS environment variable for C++11 support
ENV CXXFLAGS="-std=c++11"
ENV AWS_DEFAULT_REGION="eu-central-1"
ENV TRANSFORMERS_OFFLINE="1"
ENV SENTENCE_TRANSFORMERS_HOME="/root/.cache/huggingface/"

# Install packages
COPY requirements.txt ./
RUN python3 -m pip install -r requirements.txt

# Setup directories
RUN mkdir -p /tmp/content/pickles/ ~/.cached
RUN chmod 0700 ~/.cached

# copy all code and lambda handler
COPY *.py ./

RUN python3 dependency.py

# run lambda handler
CMD ["function.handler"]

构建错误日志 #

docker build . -t ml-server
[+] Building 6.5s (13/13) FINISHED
 => [internal] load build definition from Dockerfile                                                                   0.0s
 => => transferring dockerfile: 1.38kB                                                                                 0.0s
 => [internal] load .dockerignore                                                                                      0.0s
 => => transferring context: 32B                                                                                       0.0s
 => [internal] load metadata for public.ecr.aws/lambda/python:latest                                                   1.3s
 => [internal] load build context                                                                                      0.0s
 => => transferring context: 287B                                                                                      0.0s
 => [1/9] FROM public.ecr.aws/lambda/python:latest@sha256:d8a8324834a079dbdfc6551831325113512a147bf70003622412565f216  0.0s
 => CACHED [2/9] RUN yum update -y &&     yum install deltarpm -y &&     yum groupinstall "Development Tools" -y       0.0s
 => CACHED [3/9] COPY requirements.txt ./                                                                              0.0s
 => CACHED [4/9] RUN echo `whoami`                                                                                     0.0s
 => CACHED [5/9] RUN python3 -m pip install -r requirements.txt                                                        0.0s
 => [6/9] RUN mkdir -p /tmp/content/pickles/ ~/.cached                                                                 0.3s
 => [7/9] RUN chmod 0700 ~/.cached                                                                                     0.2s
 => [8/9] COPY *.py ./                                                                                                 0.0s
 => ERROR [9/9] RUN python3 dependency.py                                                                              4.6s
------
 > [9/9] RUN python3 dependency.py:
#13 1.260 INFO:numexpr.utils:NumExpr defaulting to 4 threads.
#13 2.806 INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: hkunlp/instructor-large
#13 2.807 DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): huggingface.co:443
#13 3.658 DEBUG:urllib3.connectionpool:https://huggingface.co:443 "GET /api/models/hkunlp/instructor-large HTTP/1.1" 200 140783
#13 4.016 Traceback (most recent call last):
#13 4.016   File "/var/task/dependency.py", line 7, in <module>
#13 4.016     instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-large",
#13 4.016                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#13 4.016   File "/var/lang/lib/python3.11/site-packages/langchain/embeddings/huggingface.py", line 150, in __init__
#13 4.016     self.client = INSTRUCTOR(
#13 4.016                   ^^^^^^^^^^^
#13 4.016   File "/var/lang/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 87, in __init__
#13 4.017     snapshot_download(model_name_or_path,
#13 4.017   File "/var/lang/lib/python3.11/site-packages/sentence_transformers/util.py", line 491, in snapshot_download
#13 4.017     path = cached_download(**cached_download_args)
#13 4.017            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#13 4.017   File "/var/lang/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
#13 4.017     return fn(*args, **kwargs)
#13 4.017            ^^^^^^^^^^^^^^^^^^^
#13 4.017   File "/var/lang/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 749, in cached_download
#13 4.018     raise LocalEntryNotFoundError(
#13 4.018 huggingface_hub.utils._errors.LocalEntryNotFoundError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.
------
executor failed running [/bin/sh -c python3 dependency.py]: exit code: 1

错误原因 #

核心问题是Dockerfile中设置了ENV TRANSFORMERS_OFFLINE="1"，该环境变量会强制transformers库进入离线模式：

• 离线模式下，库会优先查找本地缓存，不会尝试从Hugging Face Hub下载新文件
• 虽然镜像能访问Hugging Face的API（日志中返回200），但实际下载模型文件时会被离线模式阻止，导致缓存未找到错误
• 添加force_download=True无效，因为离线模式会忽略该参数

CPU瓶颈不是主要原因，日志中下载过程仅耗时几秒，未出现超时或资源耗尽的迹象。

修复方案 #

1. 关闭离线模式 #

修改Dockerfile，移除或注释掉TRANSFORMERS_OFFLINE="1"：

# lambda base image for Docker from AWS
FROM public.ecr.aws/lambda/python:latest

# Update the package list and install development tools for C++ support
RUN yum update -y && \
    yum install deltarpm -y && \
    yum groupinstall "Development Tools" -y 

# Export CXXFLAGS environment variable for C++11 support
ENV CXXFLAGS="-std=c++11"
ENV AWS_DEFAULT_REGION="eu-central-1"
# 移除离线模式环境变量
# ENV TRANSFORMERS_OFFLINE="1"
ENV SENTENCE_TRANSFORMERS_HOME="/root/.cache/huggingface/"

# Install packages
COPY requirements.txt ./
RUN python3 -m pip install -r requirements.txt

# Setup directories
RUN mkdir -p /tmp/content/pickles/ ~/.cached ./instructor_cache ./model ./tokenizer
RUN chmod 0700 ~/.cached

# copy all code and lambda handler
COPY *.py ./

RUN python3 dependency.py

# run lambda handler
CMD ["function.handler"]

2. 优化模型下载路径（可选） #

修改dependency.py，指定模型缓存目录，确保模型文件被保存到镜像的工作目录，方便Lambda运行时加载：

import logging
logging.basicConfig(level=logging.DEBUG)
from langchain.embeddings import HuggingFaceInstructEmbeddings

from transformers import AutoModelForSequenceClassification, AutoTokenizer

# 指定缓存目录，避免使用默认的root缓存
instructor_embeddings = HuggingFaceInstructEmbeddings(
    model_name="hkunlp/instructor-large",
    model_kwargs={"device": "cpu"},
    encode_kwargs={"batch_size": 1},
    cache_folder="./instructor_cache"  # 添加缓存目录参数
)

model_id = "SamLowe/roberta-base-go_emotions"

# 下载时指定缓存目录
model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    force_download=True,
    cache_dir="./model_cache"  # 添加缓存目录
)
tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    force_download=True,
    cache_dir="./model_cache"
)

# Save the model and tokenizer to a directory
model.save_pretrained("./model")
tokenizer.save_pretrained("./tokenizer")

3. 本地预下载模型（可选，避免构建时网络依赖） #

如果构建环境网络不稳定，可以在本地提前下载模型，然后复制到镜像中：

1. 本地运行dependency.py下载模型到指定目录
2. 在Dockerfile中添加复制命令：

# 复制本地预下载的模型文件
COPY ./instructor_cache ./instructor_cache
COPY ./model ./model
COPY ./tokenizer ./tokenizer

这样构建时无需联网下载模型，直接使用本地文件。

内容的提问来源于stack exchange，提问作者Fardeen Khan