Spacy加载en_core_web_sm报AttributeError错误求助

问题 #

运行main.py时触发如下错误：

Traceback (most recent call last):
  File "/Users/tyler/Desktop/Working Folder/trending-stories/main.py", line 4, in <module>
    from news_processor import NewsProcessor
  File "/Users/tyler/Desktop/Working Folder/trending-stories/news_processor.py", line 2, in <module>
    from keywords_extraction import KeywordsExtract
  File "/Users/tyler/Desktop/Working Folder/trending-stories/keywords_extraction.py", line 12, in <module>
    class KeywordsExtract:
  File "/Users/tyler/Desktop/Working Folder/trending-stories/keywords_extraction.py", line 13, in KeywordsExtract
    MODEL = spacy.load("en_core_web_sm")
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/__init__.py", line 30, in load
    return util.load_model(name, **overrides)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/util.py", line 164, in load_model
    return load_model_from_package(name, **overrides)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/util.py", line 185, in load_model_from_package
    return cls.load(**overrides)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/en_core_web_sm/__init__.py", line 12, in load
    return load_model_from_init_py(__file__, **overrides)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/util.py", line 228, in load_model_from_init_py
    return load_model_from_path(data_path, meta, **overrides)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/util.py", line 211, in load_model_from_path
    return nlp.from_disk(model_path, exclude=disable)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/language.py", line 947, in from_disk
    util.from_disk(path, deserializers, exclude)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/util.py", line 654, in from_disk
    reader(path / key)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/language.py", line 932, in <lambda>
    ) and _fix_pretrained_vectors_name(self)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/language.py", line 1071, in _fix_pretrained_vectors_name
    proc.cfg.setdefault("deprecation_fixes", {})
AttributeError: 'getset_descriptor' object has no attribute 'setdefault'

相关keywords_extraction.py代码片段：

class KeywordsExtract:
    MODEL = spacy.load("en_core_web_sm")
    allow_types = ['PERSON', 'GPE', 'ORG', 'NORP', 'LOC', 'FAC', 'WORK_OF_ART', 'EVENT', 'LAW', 'PRODUCT']
    remove_words = ['new', 'time', 'matter', 'source', 'people', 'story', 'reuters story']
    remove_entities = ['REUTERS', 'Reuters', 'Thomson Reuters', 'CNBC']
    months = [cd.month_name[i] for i in range(1, 13)] + [cd.month_abbr[i] for i in range(1, 13)]
    lookups = Lookups()
    lemma_keep = ["data"]
    lemma_exc = MODEL.vocab.lookups.get_table("lemma_exc")
    for w in lemma_keep:
        del lemma_exc[MODEL.vocab.strings["noun"]][w]
    lookups.add_table("lemma_exc", lemma_exc)
    lookups.add_table("lemma_rules", MODEL.vocab.lookups.get_table("lemma_rules"))
    lookups.add_table("lemma_index", MODEL.vocab.lookups.get_table("lemma_index"))
    lemmatizer = Lemmatizer(lookups)

已尝试重新安装en_core_web_sm，问题仍存在，当前Python版本为3.9.2，询问如何解决，是否和Python版本有关。

解决方案 #

这个错误是spaCy版本与Python 3.9不兼容导致的，核心是spaCy部分版本在Python 3.9环境下存在内部配置处理的bug，可按以下方式解决：

• 降级spaCy到兼容版本：安装与Python 3.9适配的spaCy稳定版本（如3.2.x系列），执行命令：

  pip install spacy==3.2.4
  

  之后重新安装对应版本的模型：

  python -m spacy download en_core_web_sm
  

• 调整模型加载时机：避免在类定义阶段直接加载模型，改为在类的初始化方法中加载，减少类加载时的潜在冲突，修改keywords_extraction.py代码如下：

  class KeywordsExtract:
      allow_types = ['PERSON', 'GPE', 'ORG', 'NORP', 'LOC', 'FAC', 'WORK_OF_ART', 'EVENT', 'LAW', 'PRODUCT']
      remove_words = ['new', 'time', 'matter', 'source', 'people', 'story', 'reuters story']
      remove_entities = ['REUTERS', 'Reuters', 'Thomson Reuters', 'CNBC']
      months = [cd.month_name[i] for i in range(1, 13)] + [cd.month_abbr[i] for i in range(1, 13)]
      lemma_keep = ["data"]
  
      def __init__(self):
          self.MODEL = spacy.load("en_core_web_sm")
          self.lookups = Lookups()
          lemma_exc = self.MODEL.vocab.lookups.get_table("lemma_exc")
          for w in self.lemma_keep:
              del lemma_exc[self.MODEL.vocab.strings["noun"]][w]
          self.lookups.add_table("lemma_exc", lemma_exc)
          self.lookups.add_table("lemma_rules", self.MODEL.vocab.lookups.get_table("lemma_rules"))
          self.lookups.add_table("lemma_index", self.MODEL.vocab.lookups.get_table("lemma_index"))
          self.lemmatizer = Lemmatizer(self.lookups)
  

• 升级Python版本（可选）：若项目允许，将Python版本升级到3.10及以上，这些版本与spaCy最新版本兼容性更好，可彻底避免这类旧版本兼容问题。

内容的提问来源于stack exchange，提问作者Sycan