使用YOLOv8处理10万张图片时触发OSError: [Errno 24]报错求助

问题重现 #

使用YOLOv8对10万张图片批量预测时，执行以下代码触发报错：

model = YOLO(self.weightpath)
src_dir = self.src_Dir
src_dir = src_dir+ '*'        
img_list = glob.glob(src_dir)
results = model(img_list, max_det = 1)

错误栈：

Traceback (most recent call last):
  File "/home/ec2-user/Deployment/main/Quality_check.py", line 63, in <module>
  File "/home/ec2-user/Deployment/utils/save_img_bins.py", line 19, in run
  File "/home/ec2-user/.local/lib/python3.7/site-packages/ultralytics/yolo/engine/model.py", line 102, in __call__
  File "/home/ec2-user/.local/lib/python3.7/site-packages/ultralytics/yolo/engine/model.py", line 202, in predict
  File "/home/ec2-user/.local/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
  File "/home/ec2-user/.local/lib/python3.7/site-packages/ultralytics/yolo/engine/predictor.py", line 116, in __call__
  File "/home/ec2-user/.local/lib/python3.7/site-packages/ultralytics/yolo/engine/predictor.py", line 147, in stream_inference
  File "/home/ec2-user/.local/lib/python3.7/site-packages/ultralytics/yolo/engine/predictor.py", line 135, in setup_source
  File "/home/ec2-user/.local/lib/python3.7/site-packages/ultralytics/yolo/data/build.py", line 164, in load_inference_source
  File "/home/ec2-user/.local/lib/python3.7/site-packages/ultralytics/yolo/data/build.py", line 148, in check_source
  File "/home/ec2-user/.local/lib/python3.7/site-packages/ultralytics/yolo/data/dataloaders/stream_loaders.py", line 339, in autocast_list
  File "/home/ec2-user/.local/lib/python3.7/site-packages/PIL/Image.py", line 3227, in open
OSError: [Errno 24] Too many open files: '/home/ec2-user/car/1016780737.jpg'

错误原因 #

Errno 24是系统级错误，说明当前进程同时打开的文件数超过了操作系统的文件描述符上限。一次性将10万张图片路径传给YOLOv8的model()方法时，内部会尝试批量加载大量图片，导致打开的文件数突破限制。

解决方案 #

1. 分批处理图片（推荐） #

将图片列表拆分为小批次，每次处理一批后释放资源，从根源上控制同时打开的文件数量：

model = YOLO(self.weightpath)
src_dir = self.src_Dir
src_dir = src_dir + '*'        
img_list = glob.glob(src_dir)

# 自定义每批处理的图片数量，根据系统配置调整，比如100-500
batch_size = 200

for start_idx in range(0, len(img_list), batch_size):
    # 截取当前批次的图片路径
    batch_imgs = img_list[start_idx:start_idx + batch_size]
    # 执行预测
    results = model(batch_imgs, max_det=1)
    
    # 在这里添加你的结果处理逻辑（如保存标注、提取检测框等）
    
    # 显式释放资源，帮助GC回收
    del results

2. 临时调高系统文件描述符限制 #

如果需要临时提升单进程的文件打开上限，可以在终端执行：

ulimit -n 65535

该命令仅对当前终端会话有效，重启后失效。

3. 永久修改系统文件描述符限制 #

若需要长期生效，可修改系统配置文件：

• 编辑/etc/security/limits.conf，添加以下内容：

  ec2-user soft nofile 65535
  ec2-user hard nofile 65535
  

• 编辑/etc/pam.d/common-session，确保包含：

  session required pam_limits.so
  

• 重启系统后生效。

注意：修改系统限制需要管理员权限，且过高的限制可能影响系统稳定性，优先推荐分批处理方案。

内容的提问来源于stack exchange，提问作者Hitesh