Python3+asyncio代码运行正常但退出时触发Bad File Descriptor错误

代码可正常执行并打印main函数结果，但程序退出阶段会抛出大量RuntimeError: Event loop is closed和OSError: [Errno 9] Bad file descriptor异常。添加asyncio.sleep(10)会将异常延迟到程序退出时触发。

代码示例 #

from pathlib import Path
import asyncio
from tqdm.asyncio import tqdm_asyncio
from lib import get_gcs_fs, run, get_gcs_folder

async def check_sentinel_exists(cloud_fs, sample_id, sentinel):
    exists = await cloud_fs._exists(sentinel)
    return sample_id if exists else None

async def filter_samples(cloud_fs, sample_id_with_sentinel):
    tasks = [check_sentinel_exists(cloud_fs, sample_id, sentinel) 
             for sample_id, sentinel in sample_id_with_sentinel]
    results = await tqdm_asyncio.gather(*tasks, desc="Checking files")
    return list(filter(None, results))

async def main():
    cloud_fs = get_gcs_fs()

    GCS_TSV = "..."
    GCS_PREFIX = "..."

    # Download TSV file if not exists
    if not Path("/tmp/rows.tsv").exists():
        run(f"gcloud storage cp {GCS_TSV} /tmp/rows.tsv")

    # Prepare sample_id_with_sentinel list
    sample_id_with_sentinel = []
    for line in Path("/tmp/rows.tsv").read_text().splitlines():
        if not line.strip():
            continue
        sample_id, _ = line.split("\t")
        WORK_SENTINEL = f"{get_gcs_folder(GCS_PREFIX, sample_id)}/stages/SOME_STAGE.DONE"
        sample_id_with_sentinel.append((sample_id, WORK_SENTINEL))

    filtered_sample_ids = await filter_samples(cloud_fs, sample_id_with_sentinel)
    return filtered_sample_ids

if __name__ == "__main__":
    sample_ids = asyncio.run(main())
    print(sample_ids)

异常信息 #

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/sslproto.py", line 684, in _process_write_backlog
    self._transport.write(chunk)
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/selector_events.py", line 924, in write
    self._fatal_error(exc, 'Fatal write error on socket transport')
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/selector_events.py", line 719, in _fatal_error
    self._force_close(exc)
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/selector_events.py", line 731, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/base_events.py", line 745, in call_soon
    self._check_closed()
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f7a32eca5c0>
transport: <_SelectorSocketTransport closing fd=25>
Traceback (most recent call last):
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/selector_events.py", line 918, in write
    n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/sslproto.py", line 684, in _process_write_backlog
    self._transport.write(chunk)
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/selector_events.py", line 924, in write
    self._fatal_error(exc, 'Fatal write error on socket transport')
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/selector_events.py", line 719, in _fatal_error
    self._force_close(exc)
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/selector_events.py", line 731, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/base_events.py", line 745, in call_soon
    self._check_closed()
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f7a32eca770>
transport: <_SelectorSocketTransport closing fd=29>
Traceback (most recent call last):
  File "/home/ved/micromamba/envs/dev/lib/python3.10/asyncio/selector_events.py", line 918, in write
    n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor

• 检查GCS客户端的资源清理逻辑：cloud_fs（来自get_gcs_fs()）大概率是异步GCS客户端，程序退出时可能未正确关闭其底层连接池、事件循环绑定的资源。asyncio.run()结束后事件循环已关闭，但客户端还在尝试发送SSL/TLS关闭报文，导致报错。需确认客户端是否有close()或aclose()这类异步关闭方法，在main函数返回前调用并等待完成。
• 避免直接调用私有方法：代码中直接调用cloud_fs._exists()私有方法，内部可能存在未正确处理的异步资源。换成客户端提供的公开异步检查方法（比如exists()），这类方法通常会正确管理连接生命周期。
• 验证连接池的闲置超时设置：如果GCS客户端使用连接池，可能存在闲置连接未及时回收的情况。调整连接池的闲置超时参数，让连接在任务完成后尽快关闭，避免事件循环关闭后还有活跃连接。
• 手动管理事件循环（替代asyncio.run()）：尝试手动创建事件循环，运行main()后，显式等待客户端资源清理完成再关闭循环。示例：

  if __name__ == "__main__":
      loop = asyncio.new_event_loop()
      try:
          sample_ids = loop.run_until_complete(main())
          print(sample_ids)
          # 添加客户端清理逻辑，比如 await cloud_fs.aclose()（如果有）
          loop.run_until_complete(asyncio.sleep(0.5))  # 给资源清理留缓冲时间
      finally:
          loop.close()
  

• 检查tqdm_asyncio的影响：tqdm_asyncio.gather()可能会额外创建异步任务，这些任务在gather完成后是否有残留？换成原生asyncio.gather()测试，排除进度条工具的干扰。
• 查看run()函数的同步阻塞影响：run(f"gcloud storage cp ...")是同步调用，可能会阻塞事件循环导致后续异步任务的连接状态异常？换成异步的文件下载方式（比如用GCS客户端的异步下载API），避免同步操作打乱事件循环时序。

内容的提问来源于stack exchange，提问作者Foobar