我之前在Linux服务器的多进程环境下处理大量高分辨率图像时，也碰到过一模一样的Matplotlib内存泄漏问题。你的代码里有几个关键的资源清理误区，再加上Linux和Windows多进程机制的差异，才导致了这个问题。下面是具体的修复方案：

核心问题分析 #

1. 后端适配问题：Windows上Matplotlib默认用的是GUI友好的后端，但Ubuntu HPC通常没有图形界面，交互式后端会残留大量未释放的GUI相关资源。
2. 错误的资源清理逻辑：你用的plt.figure().clear()反而会新建一个空的figure，不仅没清理旧资源，还额外增加了内存开销；无参数的plt.clf()和plt.close()也无法精准释放你创建的那个特定fig对象。
3. Linux多进程的fork特性：Linux默认用fork创建子进程，会直接继承父进程的Matplotlib上下文，导致资源被多个进程共享，无法正常回收。

具体修复步骤 #

1. 强制使用非交互式后端 #

在导入pyplot之前，先设置Matplotlib使用Agg后端——这是专门为无GUI环境设计的渲染后端，完全避免了GUI相关的资源泄漏：

import matplotlib
matplotlib.use('Agg')  # 必须放在import matplotlib.pyplot之前执行
import matplotlib.pyplot as plt

2. 精准清理Figure资源 #

在mark_points函数里，直接针对你创建的fig对象做彻底清理，并且手动触发垃圾回收（Python的自动GC可能不会及时回收大内存对象）：

def mark_points(im, df, file_name):
    dpi = 100
    height, width, nbands = im.shape
    fig_size = width / float(dpi), height / float(dpi)
    
    # 显式创建figure并绑定到变量
    fig = plt.figure(figsize=fig_size)
    ax = fig.add_axes([0, 0, 1, 1])
    ax.axis('off')
    ax.imshow(im, interpolation='nearest')
    sns.scatterplot(x='X', y='Y', data=df, hue='Class', ax=ax, s=2, edgecolor=None, legend=False)
    ax.set(xlim=[0, width], ylim=[height, 0], aspect=1)
    
    fig.savefig(file_name, dpi=dpi, transparent=True)
    
    # 彻底清理资源的正确步骤
    fig.delaxes(ax)  # 先移除axes引用
    plt.close(fig)   # 关闭指定的figure
    import gc
    gc.collect()     # 手动触发垃圾回收，释放大内存对象

注意：不要再调用plt.figure().clear()或者无参数的plt.close()，只针对你创建的fig对象操作。

3. 优化多进程结构 #

你当前的run函数每次循环都创建新的进程组，不仅效率低，还容易导致资源累积。改用multiprocessing.Pool管理进程，让每个进程处理固定数量的任务，利用进程池的资源隔离特性避免fork带来的上下文继承问题：

def run(output_dir, num_processes=1, multi_process=False ):
    os.makedirs(output_dir, exist_ok=True)
    # 生成所有任务的参数列表，对应原来的n=100
    tasks = [(output_dir, i) for i in range(100)]
    
    if multi_process:
        with mp.Pool(num_processes) as pool:
            pool.starmap(annotate_images, tasks)
        print('All Processes finished!!!')
    else:
        for task in tasks:
            annotate_images(*task)

修改后的完整代码 #

import os
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import pandas as pd
import multiprocessing as mp
import numpy as np
import seaborn as sns
import gc

sns.set_palette('bright')

def mark_points(im, df, file_name):
    dpi = 100
    height, width, nbands = im.shape
    fig_size = width / float(dpi), height / float(dpi)
    
    fig = plt.figure(figsize=fig_size)
    ax = fig.add_axes([0, 0, 1, 1])
    ax.axis('off')
    ax.imshow(im, interpolation='nearest')
    sns.scatterplot(x='X', y='Y', data=df, hue='Class', ax=ax, s=2, edgecolor=None, legend=False)
    ax.set(xlim=[0, width], ylim=[height, 0], aspect=1)
    
    fig.savefig(file_name, dpi=dpi, transparent=True)
    
    # 彻底清理资源
    fig.delaxes(ax)
    plt.close(fig)
    gc.collect()

def annotate_images(output_dir, process_num):
    # 把原来的10000拆分成小批次，避免单进程任务过多导致内存累积
    for j in range(100):
        im_height, im_width = 3000, 3000
        file_name = 'my_file_name_{}_{}.jpg'.format(process_num, j)
        output_file = os.path.join(output_dir, file_name)
        
        num_points = 20000
        # 用更高效的方式生成DataFrame
        df = pd.DataFrame({
            'X': np.random.randint(0, im_width, size=(num_points), dtype='int32'),
            'Y': np.random.randint(0, im_height, size=(num_points), dtype='int32'),
            'Class': np.random.choice(['A', 'B', 'C'], size=num_points)
        })
        
        im = np.random.randint(low=0, high=255, size=(im_height, im_width, 3), dtype=np.uint8)
        mark_points(im=im, df=df, file_name=output_file)

def run(output_dir, num_processes=1, multi_process=False ):
    os.makedirs(output_dir, exist_ok=True)
    tasks = [(output_dir, i) for i in range(100)]
    
    if multi_process:
        with mp.Pool(num_processes) as pool:
            pool.starmap(annotate_images, tasks)
        print('All Processes finished!!!')
    else:
        for task in tasks:
            annotate_images(*task)

if __name__ == '__main__':
    num_processes = 4
    multi_process = True
    home_dir = 'Output'
    for i in range(3):
        run(output_dir=home_dir, num_processes=num_processes, multi_process=multi_process)

额外建议 #

• 如果处理的图像数量极大，可以让每个进程处理固定数量的图像后自动退出，避免长时间运行导致的内存缓慢累积。
• 可以设置plt.rcParams['figure.max_open_warning'] = 0屏蔽警告，但这只是治标，不能解决根本的内存泄漏问题。

你遇到的报错信息：

image_annotate_cells.py:44: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
plt.figure().clear()
image_annotate_cells.py:22: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
fig = plt.figure(figsize=fig_size)
Process Process-1:
Traceback (most recent call last):
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "image_annotate_cells.py", line 74, in annotate_images
    mark_points(im=im, df=df, file_name=output_file)
  File "image_annotate_cells.py", line 42, in mark_points
    fig.savefig(file_name, dpi=dpi, transparent=True)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/figure.py", line 2203, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/backend_bases.py", line 2126, in print_figure
    **kwargs)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/cbook/deprecation.py", line 358, in wrapper
    return func(*args, **kwargs)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py", line 584, in print_jpg
    FigureCanvasAgg.draw(self)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py", line 393, in draw
    self.figure.draw(self.renderer)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/figure.py", line 1736, in draw
    renderer, self, artists, self.suppressComposite)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/image.py", line 137, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 2630, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/image.py", line 137, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/collections.py", line 894, in draw
    Collection.draw(self, renderer)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/collections.py", line 369, in draw
    self._offset_position)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/site-packages/matplotlib/path.py", line 197, in vertices
    @property
KeyboardInterrupt
Traceback (most recent call last):
  File "image_annotate_cells.py", line 124, in <module>
    run(**params)
  File "image_annotate_cells.py", line 97, in run
    p.join()
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/multiprocessing/process.py", line 124, in join
    res = self._popen.wait(timeout)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/multiprocessing/popen_fork.py", line 50, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
  File "/home/DMP/DUDMP/COPAINGE/yhagos/.conda/envs/tf2cpu/lib/python3.6/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt

内容的提问来源于stack exchange，提问作者Shemamu