运行emnlp22-re3-story-generation代码遇ModuleNotFoundError求助

问题背景 #

运行emnlp22-re3-story-generation仓库的快速启动notebook时，执行指定代码单元后出现上述错误，已通过pip安装huggingface_hub。

执行的代码单元 #

!CUDA_VISIBLE_DEVICES=0 python -u ../scripts/main.py --summarizer gpt3_summarizer --controller longformer_classifier longformer_classifier --loader alignment coherence --controller-load-dir emnlp22_re3_data/ckpt/relevance_reranker emnlp22_re3_data/ckpt/coherence_reranker --controller-model-string allenai/longformer-base-4096 allenai/longformer-base-4096 --save-outline-file output/outline0.pkl --save-complete-file output/complete_story0.pkl --log-file output/story0.log

完整报错信息 #

Traceback (most recent call last):
  File "../scripts/main.py", line 20, in <module>
    from story_generation.edit_module.entity import *
  File "/home/jovyan/emnlp22-re3-story-generation/story_generation/edit_module/entity.py", line 20, in <module>
    from story_generation.common.util import *
  File "/home/jovyan/emnlp22-re3-story-generation/story_generation/common/util.py", line 13, in <module>
    from sentence_transformers import SentenceTransformer
  File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/__init__.py", line 3, in <module>
    from .datasets import SentencesDataset, ParallelSentencesDataset
  File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/datasets/__init__.py", line 3, in <module>
    from .ParallelSentencesDataset import ParallelSentencesDataset
  File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/datasets/ParallelSentencesDataset.py", line 4, in <module>
    from .. import SentenceTransformer
  File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/SentenceTransformer.py", line 25, in <module>
    from .evaluation import SentenceEvaluator
  File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/evaluation/__init__.py", line 5, in <module>
    from .InformationRetrievalEvaluator import InformationRetrievalEvaluator
  File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/evaluation/InformationRetrievalEvaluator.py", line 6, in <module>
    from ..util import cos_sim, dot_score
  File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/util.py", line 407, in <module>
    from huggingface_hub.snapshot_download import REPO_ID_SEPARATOR
ModuleNotFoundError: No module named 'huggingface_hub.snapshot_download'

解决方法 #

• 升级huggingface_hub到兼容版本
  错误源于安装的huggingface_hub版本过低，REPO_ID_SEPARATOR在旧版本中不属于snapshot_download模块。执行以下命令升级：

  pip install --upgrade huggingface_hub
  

• 匹配sentence-transformers与huggingface_hub版本
  若升级后仍报错，可能是sentence-transformers版本和huggingface_hub不兼容。建议安装稳定兼容的sentence-transformers版本：

  pip install sentence-transformers==2.2.2
  

• 验证环境一致性
  执行以下命令确认huggingface_hub的安装路径和版本，确保是当前notebook使用的Python环境：

  pip show huggingface_hub
  

  如果路径不是/opt/conda/lib/python3.8/site-packages/，说明安装到了其他环境，需在当前环境重新安装或升级。

内容的提问来源于stack exchange，提问作者albusdemens