使用PyTorch下载FashionMNIST遇‘File not found or corrupted’错误求助

环境信息 #

• Python版本：3.9
• PyTorch版本：2.0.1

运行代码 #

import torch
import torchvision
from torch.utils import data
from torchvision import transforms
trans = transforms.ToTensor()
mnist_train = torchvision.datasets.FashionMNIST(root='../data', train=True, transform=trans, download=True)
mnist_test = torchvision.datasets.FashionMNIST(root='../data', train=False, transform=trans, download=True)

报错日志 #

RuntimeError                              Traceback (most recent call last)
/home/ds_ghr/code/DL/train.ipynb Cell 22 in ()
      1 trans = transforms.ToTensor()
----> 2 mnist_train = torchvision.datasets.FashionMNIST(root='../data', train=True, transform=trans, download=True)
      3 mnist_test = torchvision.datasets.FashionMNIST(root='../data', train=False, transform=trans, download=True)
      4 len(mnist_train), len(mnist_test)

File ~/anaconda3/lib/python3.9/site-packages/torchvision/datasets/mnist.py:99, in MNIST.__init__(self, root, train, transform, target_transform, download)
     96     return
     98 if download:
---> 99     self.download()
    101 if not self._check_exists():
    102     raise RuntimeError("Dataset not found. You can use download=True to download it")

File ~/anaconda3/lib/python3.9/site-packages/torchvision/datasets/mnist.py:187, in MNIST.download(self)
    185 try:
    186     print(f"Downloading {url}")
---> 187     download_and_extract_archive(url, download_root=self.raw_folder, filename=filename, md5=md5)
    188 except URLError as error:
    189     print(f"Failed to download (trying next):\n{error}")

File ~/anaconda3/lib/python3.9/site-packages/torchvision/datasets/utils.py:434, in download_and_extract_archive(url, download_root, extract_root, filename, md5, remove_finished)
    431 if not filename:
    432     filename = os.path.basename(url)
...
    153 # check integrity of downloaded file
    154 if not check_integrity(fpath, md5):
---> 155     raise RuntimeError("File not found or corrupted.")

RuntimeError: File not found or corrupted.

解决方法 #

1. 清理损坏缓存后重新下载 #

• 定位到../data/FashionMNIST/raw目录，删除其中所有已下载的文件
• 重新运行代码，让Torchvision重新完整下载数据集

2. 手动下载并校验文件 #

• 下载FashionMNIST的四个核心文件，放到../data/FashionMNIST/raw目录下
• 校验每个文件的MD5值是否匹配：
  • train-images-idx3-ubyte.gz: 8d4fb7e6c68d591d4c3dfef9ec88bf0d
  • train-labels-idx1-ubyte.gz: 25c81989df183df01b3e8a0aad5dffbe
  • t10k-images-idx3-ubyte.gz: bef4ecab320f06d8554ea6380940ec79
  • t10k-labels-idx1-ubyte.gz: bb300cfdad3c16e7a12a480ee83cd310
• 修改代码中download=True为download=False，运行代码加载数据集

3. 排查网络与权限问题 #

• 确认网络可正常访问外部数据源，有代理的话配置好Python的代理环境（设置HTTP_PROXY和HTTPS_PROXY环境变量）
• 检查../data目录的读写权限，可执行sudo chmod -R 755 ../data赋予权限

内容的提问来源于stack exchange，提问作者HAORAN GU