Colab无法加载Hugging Face数据集mozilla-foundation/common_voice_13_0求助

问题详情 #

在Google Colab中运行代码加载Hugging Face Hub上的mozilla-foundation/common_voice_13_0数据集时出现错误，但该数据集确实存在，且本地Jupyter Lab环境中能正常加载。

运行代码 #

! pip install transformers datasets
from datasets import load_dataset
cv_13 = load_dataset("mozilla-foundation/common_voice_13_0", "en", split="train")

报错信息 #

<ipython-input-9-4d772f75be89> in <cell line: 3>()
      1 from datasets import load_dataset
      2 
----> 3 cv_13 = load_dataset("mozilla-foundation/common_voice_13_0", "en", split="train")

2 frames
/usr/local/lib/python3.10/dist-packages/datasets/load.py in dataset_module_factory(path, revision, download_config, download_mode, dynamic_modules_path, data_dir, data_files, **download_kwargs)
   1505                     raise e1 from None
   1506                 if isinstance(e1, FileNotFoundError):
-> 1507                     raise FileNotFoundError(
   1508                         f"Couldn't find a dataset script at {relative_to_absolute_path(combined_path)} or any data file in the same directory. "
   1509                         f"Couldn't find '{path}' on the Hugging Face Hub either: {type(e1).__name__}: {e1}"

FileNotFoundError: Couldn't find a dataset script at /content/mozilla-foundation/common_voice_13_0/common_voice_13_0.py or any data file in the same directory. Couldn't find 'mozilla-foundation/common_voice_13_0' on the Hugging Face Hub either: FileNotFoundError: Dataset 'mozilla-foundation/common_voice_13_0' doesn't exist on the Hub. If the repo is private or gated, make sure to log in with `huggingface-cli login`.

解决步骤 #

• 更新datasets库到最新版本：Colab默认预装的datasets版本可能较旧，无法适配Hub的最新数据集结构或API。执行以下命令升级：

  !pip install --upgrade datasets
  

• 登录Hugging Face Hub：即使数据集是公开的，Colab环境的API调用可能需要身份验证才能正常访问Hub资源。运行登录命令：

  !huggingface-cli login
  

  按照提示输入你的Hugging Face访问令牌（在账号设置页面生成并复制即可）。
• 指定数据集版本分支：如果数据集有版本更新，显式指定revision参数为main分支，确保加载最新的数据集脚本：

  cv_13 = load_dataset("mozilla-foundation/common_voice_13_0", "en", split="train", revision="main")
  

• 重启Colab运行时：临时的网络或环境缓存问题可能导致加载失败，重启运行后重新执行代码即可。

内容的提问来源于stack exchange，提问作者Mohammad Moosavi