调用垃圾回收器后，音乐分类Librosa特征提取代码仍内存泄漏

算力阿峰

2026-6-19

音乐流派分类模型内存泄漏排查

我们正在为大学毕业设计开发音乐流派分类机器学习模型，使用Librosa库提取音频特征。编写的代码存在内存泄漏问题，即使在函数调用中手动触发垃圾回收器也无法解决，附上代码恳请排查泄漏原因。

注：代码最初在VS Code的Jupyter Notebook扩展中编写，后转为py文件。

原始代码

import os
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import librosa
import librosa.display
import soundfile as sf
from glob import glob
from itertools import cycle
import random
import gc
import queue

# Seaborn visualization setup
sns.set_theme(style="white", palette=None)
color_pal = plt.rcParams["axes.prop_cycle"].by_key()["color"]
color_cycle = cycle(plt.rcParams["axes.prop_cycle"].by_key()["color"])

# %%
# Path to the nested folder structure
# audio_files = glob('genres_original/**/*.wav')
# print(audio_files)

audio_files = list()
subdirectories = os.listdir("./genres_original")
for subdirectory in subdirectories:
    subdirectoryPath = os.path.join("./genres_original",subdirectory)
    if os.path.isdir(subdirectoryPath):
        files = os.listdir(subdirectoryPath)
        selected_files = random.sample(files,1)
        full_path_selected_files = [os.path.join(subdirectoryPath,wav_file) for wav_file in selected_files]
        audio_files.extend(full_path_selected_files)
print(audio_files[:10])

# Output directories for saving plots and MFCC features
output_dir = 'output'
spectrogram_dir = os.path.join(output_dir, 'spectrogram_plots')
mel_spectrogram_dir = os.path.join(output_dir, 'mel_spectrogram_plots')
mfcc_dir = os.path.join(output_dir, 'mfcc_features')
os.makedirs(spectrogram_dir, exist_ok=True)
os.makedirs(mel_spectrogram_dir, exist_ok=True)
os.makedirs(mfcc_dir, exist_ok=True)

# %%
# Function to augment audio
def augment_audio(y):
    y_pitch_shifted = librosa.effects.pitch_shift(y, sr=22050, n_steps=4)  # Assuming default sr
    y_time_stretched = librosa.effects.time_stretch(y=y, rate=1.5)
    noise = np.random.randn(len(y))
    y_noisy = y + 0.005 * noise
    listReturn = [y_pitch_shifted, y_time_stretched, y_noisy]
    return listReturn

# %%
def plotSTFT(y, sr, file_basename, suffix):
    # Compute and save STFT spectrogram plot
    D = librosa.stft(y)
    S_db = librosa.amplitude_to_db(np.abs(D), ref=np.max)
    fig, ax = plt.subplots(figsize=(8, 4))  # Reduced figure size
    img = librosa.display.specshow(S_db, x_axis='time', y_axis='log', ax=ax)
    ax.set_title(f'Spectrogram Waveform of {file_basename}{suffix}', fontsize=14)
    fig.colorbar(img, ax=ax, format='%0.2f')
    spectrogram_filename = os.path.join(spectrogram_dir, f'{file_basename}{suffix}.png')
    plt.savefig(spectrogram_filename)
    plt.close(fig)
    del fig,spectrogram_filename,D,S_db,ax,img
    gc.collect()

# %%
def plotMelSpectrogram(y, sr, file_basename, suffix):
    # Compute and save Mel spectrogram plot
    S = librosa.feature.melspectrogram(y=y, sr=sr, n_mels=128)
    S_db_mel = librosa.amplitude_to_db(S, ref=np.max)
    fig, ax = plt.subplots(figsize=(8, 4))  # Reduced figure size
    img = librosa.display.specshow(S_db_mel, x_axis='time', y_axis='log', ax=ax)
    ax.set_title(f'Mel Spectrogram Waveform of {file_basename}{suffix}', fontsize=14)
    fig.colorbar(img, ax=ax, format='%0.2f')
    mel_spectrogram_filename = os.path.join(mel_spectrogram_dir, f'{file_basename}{suffix}.png')
    plt.savefig(mel_spectrogram_filename)
    plt.close(fig)
    del fig,mel_spectrogram_filename,S,S_db_mel,ax,img
    gc.collect()

# %%
def computeMFCC(y, sr, file_basename, suffix):
    # Compute and save MFCC features
    mfccs = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=13)
    mfcc_filename = os.path.join(mfcc_dir, f'{file_basename}{suffix}.csv')
    mfcc_df = pd.DataFrame(mfccs)
    mfcc_df.to_csv(mfcc_filename, index=False)
    del mfccs, mfcc_filename, mfcc_df
    gc.collect()

# %%
# Function to process a single audio file
def process_audio_file(y, sr, file_basename, augment_index=None):
    suffix = f'aug{augment_index}' if augment_index is not None else ''
    plotSTFT(y, sr, file_basename, suffix)
    plotMelSpectrogram(y, sr, file_basename, suffix)
    computeMFCC(y, sr, file_basename, suffix)
    gc.collect()

# %%
def doStuff(wav_file):
    file_basename = os.path.splitext(os.path.basename(wav_file))[0]
    # Load the audio file
    y, sr = librosa.load(wav_file)
    
    # Process the original audio file
    process_audio_file(y, sr, file_basename)
    # Augment the audio and process augmented versions
    augmented_audios = augment_audio(y)
    for i, aug_y in enumerate(augmented_audios):
        augmented_filename = os.path.join(output_dir, f'{file_basename}aug{i}.wav')
        sf.write(augmented_filename, aug_y, sr)  # Save the augmented audio file
        process_audio_file(aug_y, sr, file_basename, augment_index=i)
        os.remove(augmented_filename)
    del augmented_audios, y, sr, file_basename
    gc.collect()

# %%

fileQueue = queue.Queue()
for wav_file in audio_files:
    fileQueue.put(wav_file)

while not fileQueue.empty():
    wav_file = fileQueue.get()
    doStuff(wav_file)
    gc.collect()

泄漏原因分析及修复方案

1. Matplotlib 后端资源泄漏

默认的Matplotlib交互式后端（如TkAgg、QtAgg）会保留figure的隐式引用，即使调用plt.close(fig)也无法彻底释放内存。

修复：
在导入matplotlib后立即切换到非交互式后端Agg，避免GUI相关资源占用：

import matplotlib
matplotlib.use('Agg')  # 放在import matplotlib.pyplot之前
import matplotlib.pyplot as plt

2. Librosa 内部缓存累积

Librosa的部分函数会缓存中间计算结果，处理大量文件时缓存会持续占用内存，且不会被Python垃圾回收自动清理。

修复：
在每个音频文件处理完成后，手动清理Librosa缓存：

def doStuff(wav_file):
    # ... 原有代码 ...
    del augmented_audios, y, sr, file_basename
    librosa.cache.clear()  # 添加这行清理Librosa缓存
    gc.collect()

3. Pandas DataFrame 额外内存开销

computeMFCC中用Pandas DataFrame保存MFCC特征，DataFrame内部的元数据会产生额外内存占用，且销毁时可能不如numpy数组彻底。

修复：
直接用numpy保存MFCC数组，跳过DataFrame转换：

def computeMFCC(y, sr, file_basename, suffix):
    mfccs = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=13)
    mfcc_filename = os.path.join(mfcc_dir, f'{file_basename}{suffix}.csv')
    np.savetxt(mfcc_filename, mfccs, delimiter=',')  # 替换DataFrame保存逻辑
    del mfccs, mfcc_filename
    gc.collect()

4. 队列元素引用残留

处理队列中的文件路径后，变量wav_file的引用可能未及时释放。

修复：
在处理完文件后显式删除该变量：

while not fileQueue.empty():
    wav_file = fileQueue.get()
    doStuff(wav_file)
    del wav_file  # 显式释放文件路径引用
    gc.collect()

额外优化建议

避免在循环内频繁调用gc.collect()，仅在每个文件处理完成后调用一次即可，过度调用会降低运行效率。
可以使用memory_profiler库（pip install memory-profiler）来定位具体的内存泄漏点，通过@profile装饰器标记需要监控的函数，运行时用python -m memory_profiler your_script.py查看内存变化。

内容的提问来源于stack exchange，提问作者RACHIT MITTAL