PyInstaller打包exe运行报错：无法导入transformers.generation的GenerationMixin

运行打包后的exe文件时出现如下报错：

cannot import name 'GenerationMixin' from 'transformers.generation' (C:\Users\UserName\AppData\Local\Temp\_MEI198962\transformers\generation\__init__.pyc)

当前使用transformers版本为4.26.1，尝试降级至4.25.1后问题仍未解决。以下是相关代码、依赖配置及spec文件：

核心代码 #

model = pickle.load(open(r"miniLM.sav", "rb")) 
sentences_embeddings = model.encode(desc_corpus) 
c_matrix = cosine_similarity(sentences_embeddings, sentences_embeddings)

requirements.txt #

absl-py==1.4.0
altgraph==0.17.3
astunparse==1.6.3
cachetools==5.3.0
certifi==2022.12.7
charset-normalizer==3.0.1
click==8.1.3
colorama==0.4.6
et-xmlfile==1.1.0
filelock==3.9.0
flatbuffers==23.1.21
gast==0.4.0
google-auth==2.16.0
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.51.1
h5py==3.8.0
huggingface-hub==0.12.0
idna==3.4
importlib-metadata==6.0.0
joblib==1.2.0
keras==2.11.0
libclang==15.0.6.1
Markdown==3.4.1
MarkupSafe==2.1.2
nltk==3.8.1
numpy==1.24.2
oauthlib==3.2.2
openpyxl==3.1.0
opt-einsum==3.3.0
packaging==23.0
pandas==1.5.3
pefile==2023.2.7
Pillow==9.4.0
protobuf==3.19.6
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyinstaller==5.7.0
pyinstaller-hooks-contrib==2022.15
PyQt5==5.15.9
PyQt5-Qt5==5.15.2
PyQt5-sip==12.11.1
python-dateutil==2.8.2
python-version==0.0.2
pytz==2022.7.1
pywin32-ctypes==0.2.0
PyYAML==6.0
regex==2022.10.31
requests==2.28.2
requests-oauthlib==1.3.1
rsa==4.9
scikit-learn==1.2.1
scipy==1.10.0
sentence-transformers==2.2.2
sentencepiece==0.1.97
six==1.16.0
tensorboard==2.11.2
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.11.0
tensorflow-estimator==2.11.0
tensorflow-intel==2.11.0
tensorflow-io-gcs-filesystem==0.30.0
termcolor==2.2.0
threadpoolctl==3.1.0
tokenizers==0.13.2
torch==1.13.1
torchvision==0.14.1
tqdm==4.64.1
transformers==4.26.1
typing_extensions==4.4.0
urllib3==1.26.14
Werkzeug==2.2.2
wrapt==1.14.1
zipp==3.13.0

spec文件 #

# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import copy_metadata

datas = [('Config\\favicon.ico', '.'), ('Config\\miniLM.sav', '.')]
datas += copy_metadata('tqdm')
datas += copy_metadata('regex')
datas += copy_metadata('requests')
datas += copy_metadata('packaging')
datas += copy_metadata('filelock')
datas += copy_metadata('numpy')
datas += copy_metadata('tokenizers')
datas += copy_metadata('importlib_metadata')
datas += copy_metadata('tensorflow')


block_cipher = None


a = Analysis(
    ['render_ui.py'],
    pathex=[],
    binaries=[],
    datas=datas,
    hiddenimports=['sklearn.metrics._pairwise_distances_reduction._datasets_pair', 'sklearn.metrics._pairwise_distances_reduction._middle_term_computer', 'sklearn.metrics._pairwise_distances_reduction._argkmin', 'sklearn.metrics._pairwise_distances_reduction._base', 'sklearn.metrics._pairwise_distances_reduction._radius_neighbors', 'sentence_transformers.SentenceTransformer', 'tensorflow'],
    hookspath=[],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    win_no_prefer_redirects=False,
    win_private_assemblies=False,
    cipher=block_cipher,
    noarchive=False,
)
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)

exe = EXE(
    pyz,
    a.scripts,
    a.binaries,
    a.zipfiles,
    a.datas,
    [],
    name='App',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    upx_exclude=[],
    runtime_tmpdir=None,
    console=True,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
    icon='Config\favicon.ico'
)

解决方案 #

1. 对齐模型训练与打包的依赖版本
   你用pickle加载的miniLM.sav是训练时保存的模型，需确保打包时的transformers和sentence-transformers版本与训练时完全一致。比如训练时若用的是transformers 4.22.x，就降级到对应版本，而非随意更换版本。

2. 补充PyInstaller的hiddenimports
   修改spec文件的hiddenimports，添加transformers的缺失模块：

   hiddenimports=[
       # 原有导入...
       'transformers.generation.GenerationMixin',
       'transformers.generation.utils',
       'transformers.generation'
   ],
   

   PyInstaller经常无法自动捕获transformers的内部子模块导入，手动添加可解决缺失问题。

3. 清理缓存后重新打包

   • 删除当前生成的build、dist文件夹
   • 清理系统临时目录中_MEI开头的文件夹
   • 重新执行pyinstaller your_spec_file.spec

4. 改用官方方式保存/加载模型（推荐）
   避免用pickle保存sentence-transformers模型，改用官方API：

   # 保存模型时
   model.save('miniLM_model')
   # 加载模型时
   from sentence_transformers import SentenceTransformer
   model = SentenceTransformer('miniLM_model')
   

   这种方式保存的模型包含完整结构，PyInstaller能更好识别依赖，同时避免版本兼容问题。

内容的提问来源于stack exchange，提问作者abhiram subramanya