Django部署出现KeyError:118错误，请求协助排查修复

错误根源分析 #

这个KeyError: 118是pickle反序列化时的环境兼容问题：即使你在Docker同环境训练模型，仍可能存在依赖版本（如numpy、joblib、scikit-learn）的细微差异；同时全局加载模型的方式会在Django启动阶段触发加载，容易因环境初始化不完全引发错误。

具体修复步骤 #

1. 统一依赖版本并固化 #

确保训练环境与部署环境的核心依赖版本完全一致：

• 在模型训练环境生成精确依赖清单：

  pip freeze > requirements.txt
  

• 在Dockerfile中指定安装该依赖文件：

  COPY requirements.txt .
  RUN pip install --no-cache-dir -r requirements.txt
  

2. 延迟模型加载到视图内部 #

将全局加载模型改为按需加载，避免Django启动时的初始化问题：
修改views.py如下：

from django.shortcuts import render
from joblib import load
import requests
import os
from django.conf import settings
from django.views.decorators.cache import never_cache

# 初始化模型变量为None
movies_list = None
similarity = None

def get_models():
    """按需加载模型，确保只加载一次"""
    global movies_list, similarity
    if movies_list is None or similarity is None:
        movies_list_path = os.path.join(settings.BASE_DIR, 'savedmodels/movies.joblib')
        model_path = os.path.join(settings.BASE_DIR, 'savedmodels/similarity.joblib')
        movies_list = load(open(movies_list_path, 'rb'))
        similarity = load(open(model_path,'rb'))
    return movies_list, similarity

def home(request):
    movies_list, _ = get_models()
    return render(request, 'index.html', {'movies_list': movies_list['title'].values})

def fetch_poster(movie_id):
    response=requests.get('https://api.themoviedb.org/3/movie/{}?api_key=fbb18fe9735d1a3e0ad1531398ed1b13'.format(movie_id))
    data=response.json()
    return 'https://image.tmdb.org/t/p/w500/'+ data['poster_path']

def recommend(movie):
    movies_list, similarity = get_models()
    movie_index = movies_list[movies_list['title'] == movie].index[0]
    distances = similarity[movie_index]
    movie_list = sorted(list(enumerate(distances)), reverse=True, key=lambda x: x[1])[1:7]
    recommended_movies = []
    recommended_movies_posters = []
    for i in movie_list:
        movie_id = movies_list.iloc[i[0]].id
        recommended_movies.append(movies_list.iloc[i[0]].title)
        recommended_movies_posters.append(fetch_poster(movie_id))
    return recommended_movies, recommended_movies_posters

@never_cache
def recommend_movies(request):
    if request.method == 'POST':
        selected_movie_name = request.POST['movie_name']
        names, posters = recommend(selected_movie_name)
        movies_list, _ = get_models()
        return render(request, 'recommend.html', {'movie_data': zip(names, posters), 'movies_list': movies_list['title'].values})

    movies_list, _ = get_models()
    return render(request, 'index.html', {'movies_list': movies_list['title'].values})

3. 验证Git LFS配置 #

确保大模型文件被正确追踪：

• 检查项目根目录的.gitattributes文件是否包含：

  savedmodels/*.joblib filter=lfs diff=lfs merge=lfs -text
  

• 重新提交模型文件：

  git lfs track "savedmodels/*.joblib"
  git add .gitattributes savedmodels/
  git commit -m "Track joblib models with LFS"
  git push
  

• 在Dockerfile中添加LFS安装步骤：

  RUN apt-get update && apt-get install -y git-lfs && git lfs install
  

4. 重新序列化模型 #

在训练环境中，用当前环境的joblib重新保存模型：

from joblib import dump
# 替换为你的实际模型变量
dump(movies_df, 'savedmodels/movies.joblib')
dump(similarity_matrix, 'savedmodels/similarity.joblib')

重新提交更新后的模型文件到仓库。

内容的提问来源于stack exchange，提问作者Shubha Ruidas