加载Word2Vec模型遇FileNotFoundError：缺失.npy文件问题咨询

问题详情 #

运行代码：

from gensim.models import Word2Vec
w2v_model = Word2Vec.load('E:/projectlzy/data/word2vec_withString10-100-200.model')

报错信息：

FileNotFoundError                         Traceback (most recent call last)
Input In [1], in <cell line: 2>()
      1 from gensim.models import Word2Vec
----> 2 w2v_model = Word2Vec.load('E:/projectlzy/data/word2vec_withString10-100-200.model')

File D:\Anaconda\envs\mykeras\lib\site-packages\gensim\models\word2vec.py:1141, in Word2Vec.load(cls, *args, **kwargs)
   1122 """Load a previously saved :class:`~gensim.models.word2vec.Word2Vec` model.
   1123 
   1124 See Also
   (...)
   1138 
   1139 """
   1140 try:
-> 1141     model = super(Word2Vec, cls).load(*args, **kwargs)
   1143     # for backward compatibility for `max_final_vocab` feature
   1144     if not hasattr(model, 'max_final_vocab'):

File D:\Anaconda\envs\mykeras\lib\site-packages\gensim\models\base_any2vec.py:1230, in BaseWordEmbeddingsModel.load(cls, *args, **kwargs)
   1199 @classmethod
   1200 def load(cls, *args, **kwargs):
   1201     """Load a previously saved object (using :meth:`~gensim.models.base_any2vec.BaseWordEmbeddingsModel.save`) from file.
   1202 
   1203     Also initializes extra instance attributes in case the loaded model does not include them.
   (...)
   1228 
   1229     """
-> 1230     model = super(BaseWordEmbeddingsModel, cls).load(*args, **kwargs)
   1231     if not hasattr(model, 'ns_exponent'):
   1232         model.ns_exponent = 0.75

File D:\Anaconda\envs\mykeras\lib\site-packages\gensim\models\base_any2vec.py:602, in BaseAny2VecModel.load(cls, fname_or_handle, **kwargs)
    575 @classmethod
    576 def load(cls, fname_or_handle, **kwargs):
    577     """Load a previously saved object (using :meth:`~gensim.models.base_any2vec.BaseAny2VecModel.save`) from a file.
    578 
    579     Parameters
   (...)
    600 
    601     """
-> 602     return super(BaseAny2VecModel, cls).load(fname_or_handle, **kwargs)

File D:\Anaconda\envs\mykeras\lib\site-packages\gensim\utils.py:436, in SaveLoad.load(cls, fname, mmap)
    433 compress, subname = SaveLoad._adapt_by_suffix(fname)
    435 obj = unpickle(fname)
-> 436 obj._load_specials(fname, mmap, compress, subname)
    437 logger.info("loaded %s", fname)
    438 return obj

File D:\Anaconda\envs\mykeras\lib\site-packages\gensim\utils.py:467, in SaveLoad._load_specials(self, fname, mmap, compress, subname)
    465     logger.info("loading %s recursively from %s.* with mmap=%s", attrib, cfname, mmap)
    466     with ignore_deprecation_warning():
-> 467         getattr(self, attrib)._load_specials(cfname, mmap, compress, subname)
    469 for attrib in getattr(self, '__numpys', []):
    470     logger.info("loading %s from %s with mmap=%s", attrib, subname(fname, attrib), mmap)

File D:\Anaconda\envs\mykeras\lib\site-packages\gensim\utils.py:478, in SaveLoad._load_specials(self, fname, mmap, compress, subname)
    476     val = np.load(subname(fname, attrib))['val']
    477 else:
-> 478     val = np.load(subname(fname, attrib), mmap_mode=mmap)
    480 with ignore_deprecation_warning():
    481     setattr(self, attrib, val)

File D:\Anaconda\envs\mykeras\lib\site-packages\numpy\lib\npyio.py:390, in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
    388     own_fid = False
    389 else:
-> 390     fid = stack.enter_context(open(os_fspath(file), "rb"))
    391     own_fid = True
    393 # Code to distinguish from NumPy binary files and pickles.

FileNotFoundError: [Errno 2] No such file or directory: 'E:/projectlzy/data/word2vec_withString10-100-200.model.wv.vectors.npy'

疑问 #

• .npy文件是否在生成.model文件时同步生成？
• 加载word2vec.model文件是否必须配套对应的.npy文件？
• 或是我使用的gensim版本（3.8.3）有误？

解答 #

• .npy文件是否同步生成？
  是的。用gensim的Word2Vec.save()方法保存模型时，会自动生成多个配套文件，包括.wv.vectors.npy。模型里的权重向量这类大型数组数据会单独存在二进制文件中，主.model文件只存元数据和对附属文件的引用。

• 加载是否必须配套.npy文件？
  必须。Word2Vec.load()加载时需要读取所有配套文件来恢复完整模型，缺任何一个都会触发FileNotFoundError。

• gensim 3.8.3版本是否有问题？
  这个版本本身稳定可用，不是报错的直接原因。但要注意：保存和加载模型尽量用同版本gensim，不同版本可能存在文件结构差异引发兼容问题，当前报错核心还是模型文件不完整。

解决建议 #

• 检查E:/projectlzy/data/路径下是否有完整的模型文件集，除.model外，还应有.wv.vectors.npy、.vocab等文件；
• 若文件丢失，重新用model.save('模型路径')完整保存模型，不要手动复制单个文件；
• 确保保存和加载模型的gensim版本一致，避免版本兼容问题。

内容的提问来源于stack exchange，提问作者zhaoyang li