使用Python字典列表推导式与OpenCV时，生成arr阶段卡顿耗内存求助

Hey there, let's break down what's going wrong here and get your code running smoothly!

The Root Cause #

Your list comprehension [ { 'img_nm' : fl, 'img' : cv2.imread( fl ) } for fl in files ] is trying to load every single JPG in your folder into memory at once. If you have hundreds/thousands of images, or even a few very large high-res ones, this will immediately clog up your RAM and make Python hang while it tries to cram everything into memory.

Fixes to Try #

• Switch to a generator (instant memory relief)
  Instead of using square brackets [] (which creates a full list), use parentheses () to make a generator expression. This way, images are only loaded one at a time when you iterate over it, not all upfront:

  arr = ( { 'img_nm' : fl, 'img' : cv2.imread( fl ) } for fl in files )
  

  Then you can loop through arr like this, processing each image as you go without hogging memory:

  for item in arr:
      # Do your processing on item['img'] here
      pass
  

• Process images in batches
  If you need to work with groups of images at once, split your file list into smaller batches. This keeps memory usage controlled, and you can free up memory after each batch is processed:

  import os
  import glob
  import cv2
  
  root = r'path\to\img'
  files = list(glob.iglob(os.path.join(root, '*.jpg')))
  batch_size = 15  # Adjust this based on your available RAM
  
  for start_idx in range(0, len(files), batch_size):
      batch = files[start_idx:start_idx + batch_size]
      # Load only the current batch
      batch_images = [{'img_nm': fl, 'img': cv2.imread(fl)} for fl in batch]
      
      # Process the batch here
      for img_data in batch_images:
          # Your image processing logic
          pass
      
      # Clear the batch from memory
      del batch_images
  

• Reduce image size on load
  If you don't need the full-resolution images, you can tell OpenCV to load a smaller version directly, which cuts down on memory per image drastically:

  # Loads image at 1/2 the original size
  img = cv2.imread(fl, cv2.IMREAD_REDUCED_COLOR_2)
  # Or 1/4 size: cv2.IMREAD_REDUCED_COLOR_4, 1/8: cv2.IMREAD_REDUCED_COLOR_8
  

  Alternatively, resize after loading if you need specific dimensions:

  img = cv2.imread(fl)
  img = cv2.resize(img, (640, 480))  # Adjust width/height to your needs
  

• Check for corrupted images
  Sometimes a single corrupted JPG can cause cv2.imread to hang or take forever to process. Add a safety check to skip bad files:

  def load_safely(file_path):
      try:
          img = cv2.imread(file_path)
          if img is None:
              print(f"Skipping unreadable file: {file_path}")
              return None
          return img
      except Exception as e:
          print(f"Error loading {file_path}: {str(e)}")
          return None
  
  # Use this function in your list/generator
  arr = [ {'img_nm' : fl, 'img' : load_safely(fl) } for fl in files if load_safely(fl) is not None ]
  

内容的提问来源于stack exchange，提问作者adam.hendry