在PythonAnywhere运行Sentence Transformers报错的原因及解决方法

问题 #

在PythonAnywhere运行sentence-transformers/all-MiniLM-L6-v2模型计算向量嵌入时出错，该模型在本地WSL2 Ubuntu环境可正常运行。安装命令pip install -U sentence-transformers执行成功，但运行以下代码时报错：

from sentence_transformers import SentenceTransformer
import time

def ms_now():
    return int(time.time_ns() / 1000000)

class Timer():
    def __init__(self):
        self.start = ms_now()
    
    def stop(self):
        return ms_now() - self.start

sentences = ["This is an example sentence each sentence is converted"] * 10

timer = Timer()
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
print("Model initialized", timer.stop())
for _ in range(10):
    timer = Timer()
    embeddings = model.encode(sentences)
    print(timer.stop())

报错信息：

Traceback (most recent call last):
  File "/home/DrMeir/test/test.py", line 17, in <module>
    model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
  File "/home/DrMeir/.local/lib/python3.9/site-packages/sentence_transformers/SentenceTransformer.py", line 95, in __init__
    modules = self._load_sbert_model(model_path)
  File "/home/DrMeir/.local/lib/python3.9/site-packages/sentence_transformers/SentenceTransformer.py", line 840, in _load_sbert_model
    module = module_class.load(os.path.join(model_path, module_config['path']))
  File "/home/DrMeir/.local/lib/python3.9/site-packages/sentence_transformers/models/Transformer.py", line 137, in load
    return Transformer(model_name_or_path=input_path, **config)
  File "/home/DrMeir/.local/lib/python3.9/site-packages/sentence_transformers/models/Transformer.py", line 29, in __init__
    self._load_model(model_name_or_path, config, cache_dir)
  File "/home/DrMeir/.local/lib/python3.9/site-packages/sentence_transformers/models/Transformer.py", line 49, in _load_model
    self.auto_model = AutoModel.from_pretrained(model_name_or_path, config=config, cache_dir=cache_dir)
  File "/home/DrMeir/.local/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 493, in from_pretrained
    return model_class.from_pretrained(
  File "/home/DrMeir/.local/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2903, in from_pretrained
    ) = cls._load_pretrained_model(
  File "/home/DrMeir/.local/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3061, in _load_pretrained_model
    id_tensor = id_tensor_storage(tensor) if tensor.device != torch.device("meta") else id(tensor)
RuntimeError: Expected one of cpu, cuda, xpu, mkldnn, opengl, opencl, ideep, hip, msnpu, xla, vulkan device type at start of device string: meta

已知PythonAnywhere使用torch 1.8.1+cpu，本地使用torch 2.0.1，求报错原因及解决方法。

报错原因 #

核心是torch版本兼容性问题：

• torch 1.8.1不支持meta设备类型，该特性在torch 1.10及以上版本才被引入。
• 新版transformers库（sentence-transformers的依赖库）加载模型时会触发meta设备相关逻辑，旧版torch无法识别该设备，导致运行时错误。

解决方法 #

方法1：降级依赖库版本 #

安装与torch 1.8.1兼容的旧版sentence-transformers和transformers：

# 卸载现有版本
pip uninstall -y sentence-transformers transformers
# 安装兼容版本
pip install sentence-transformers==2.2.2 transformers==4.18.0

这两个版本经过验证，能和torch 1.8.1+cpu稳定配合，不会触发meta设备相关逻辑。

方法2：升级torch版本 #

如果PythonAnywhere环境允许升级torch，执行以下命令：

pip install torch==1.13.1+cpu --index-url https://download.pytorch.org/whl/cpu

升级完成后，重新安装最新版sentence-transformers：

pip install -U sentence-transformers

选择torch 1.13.1是因为它支持meta设备，同时适配PythonAnywhere的Python 3.9环境，兼容性较好。

内容的提问来源于stack exchange，提问作者AlwaysLearning