使用dagshub.upload.Repo上传数据集遇DagsHubAPIError求助

问题场景 #

配合MLflow教程操作首个DagsHub仓库，尝试将本地数据集文件上传到远程受DVC版本控制的目录时触发错误。

执行代码 #

repo = dagshub.upload.Repo(USER_NAME,REPO_NAME)

repo.upload(local_path='data/winequality.txt',
            remote_path='data/raw/winequality.txt',
            commit_message='Added Raw Data',
            versioning='dvc')

报错信息 #

Uploading files (1) to "USER_NAME/REPO_NAME"...
---------------------------------------------------------------------------
DagsHubAPIError                           Traceback (most recent call last)
<ipython-input-49-e8d1e8493248> in <cell line: 4>()
      2 repo = dagshub.upload.Repo(USER_NAME,REPO_NAME)
      3 
----> 4 repo.upload(local_path='data/winequality.txt',
      5             remote_path='data/raw/winequality.txt',
      6             commit_message='Added Raw Data',

2 frames
/usr/local/lib/python3.10/dist-packages/dagshub/upload/wrapper.py in upload(self, local_path, commit_message, remote_path, **kwargs)
    286         else:
    287             file_to_upload = DataSet.get_file(str(local_path), remote_path)
--> 288             self.upload_files([file_to_upload], commit_message=commit_message, **kwargs)
    289 
    290     def upload_files(

/usr/local/lib/python3.10/dist-packages/dagshub/upload/wrapper.py in upload_files(self, files, directory_path, commit_message, versioning, new_branch, last_commit, force)
    375             timeout=None,
    376         )
--> 377         self._log_upload_details(data, res, files)
    378 
    379         # The ETag header contains the hash of the uploaded commit,

/usr/local/lib/python3.10/dist-packages/dagshub/upload/wrapper.py in _log_upload_details(self, data, res, files)
    413             log_message(f"Got unknown successful status code {res.status_code}")
    414         else:
--> 415             raise determine_upload_api_error(res)
    416 
    417     def _poll_mirror_up_to_date(self):

DagsHubAPIError: file missing from storage:
Required resource is missing from the storage, is '' stored in your repository DagsHub storage?

目录结构 #

• 本地磁盘：

root/
    |...data/
        |... winequality.txt

• 远程仓库：

root/
    |...data/
            |...raw/  # 该目录受DVC版本控制

解决方法 #

1. 先通过DVC完成本地跟踪与远程同步 #

DagsHub的上传工具依赖DVC的本地跟踪状态，建议先通过DVC命令完成文件的版本控制：

• 初始化本地DVC（未初始化时执行）：

  dvc init
  

• 配置DVC远程指向DagsHub存储：

  dvc remote add origin https://dagshub.com/USER_NAME/REPO_NAME.dvc
  dvc remote modify origin auth basic
  dvc remote modify origin user USER_NAME
  dvc remote modify origin password YOUR_DAGSHUB_TOKEN
  

• 跟踪本地文件并推送：

  dvc add data/winequality.txt
  git add data/winequality.txt.dvc .gitignore
  git commit -m "Track raw data with DVC"
  dvc push
  

完成后，远程仓库的data/raw目录会同步该文件的DVC跟踪记录。

2. 修正上传路径与本地文件位置 #

• 检查本地文件路径：确认data/winequality.txt没有多余空格（目录结构中显示文件名前有空格，可能导致路径识别错误）
• 先在本地创建data/raw目录，将文件移动到该目录后再上传：

  repo.upload(local_path='data/raw/winequality.txt',
              remote_path='data/raw/winequality.txt',
              commit_message='Added Raw Data',
              versioning='dvc')
  

3. 验证DagsHub认证权限 #

• 确保使用的DagsHub Token具备仓库读写权限，可在DagsHub个人设置中生成新Token
• 执行dagshub auth login完成本地认证，避免因权限不足导致文件上传失败

内容的提问来源于stack exchange，提问作者J.Kent