You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

升级Python3.10后使用boto3上传下载文件遇RuntimeError求助

Python3.10下boto3/awswrangler大文件上传下载触发RuntimeError问题

在Python3.8环境中,boto3.client.download_file()和boto3.client.upload_file()函数运行正常,但升级至Python3.10后,出现以下错误:

RuntimeError: cannot schedule new futures after interpreter shutdown

已测试boto3多个版本(1.17、1.18、1.26及最新版),均存在该问题。

报错堆栈信息

Traceback (most recent call last):
  File "/opt/.venv/lib/python3.10/site-packages/awswrangler/s3/_fs.py", line 585, in open_s3_object
    yield s3obj
  File "/opt/.venv/lib/python3.10/site-packages/awswrangler/s3/_upload.py", line 74, in upload
    s3_f.write(local_f.read())  # type: ignore[arg-type]
  File "/opt/.venv/lib/python3.10/site-packages/awswrangler/s3/_fs.py", line 550, in write
    self.flush()
  File "/opt/.venv/lib/python3.10/site-packages/awswrangler/s3/_fs.py", line 427, in flush
    self._upload_proxy.upload(
  File "/opt/.venv/lib/python3.10/site-packages/awswrangler/s3/_fs.py", line 133, in upload
    future = self._exec.submit(
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 169, in submit
    raise RuntimeError('cannot schedule new futures after '
RuntimeError: cannot schedule new futures after interpreter shutdown

已尝试的解决方案(均不满足需求)

  • 禁用线程:可解决错误,但大文件上传下载效率显著下降,希望保留线程以维持性能
  • 调用thread.join()等待线程结束:无效,仍触发相同错误
  • 使用awswrangler库:小文件可正常运行,大文件仍会触发该错误

寻求能保留线程功能、同时解决该RuntimeError的有效方案。

内容的提问来源于stack exchange,提问作者djGowda

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.06.21 04:35:04