PyAudio触发内存清理后录制音频失真问题求助

问题背景 #

我编写了一个Python脚本，通过实时监控音频流并使用移动平均法，根据设定阈值判断音频的起止点。由于脚本需7×24小时运行，为避免内存占用过高，当录制时长约达4小时（即counter超过MAX_LOOKBACK_PERIOD）时，会删除部分历史音频数据。

脚本运行正常，但触发内存清理逻辑后，保存的音频开始出现失真：清理前音频频谱图无异常，清理后频谱图出现垂直尖峰。推测是del操作耗时过长导致while循环无法跟上音频流，但不确定具体原因，恳请提供解决思路。

原脚本代码 #

def record(Jn):
  global current_levels
  global current_lens
  device_name = j_lookup[Jn]['device']
  device_index = get_index_by_name(device_name)
  audio = pyaudio.PyAudio()
  stream = audio.open(format=FORMAT, channels=CHANNELS, rate=RATE, input=True, input_device_index=device_index, frames_per_buffer=CHUNK)

  recorded_frames = []
  quantized_history = []
  long_window = int(LONG_MOV_AVG_SECS*RATE/CHUNK) # converting seconds to while loop counter

  avg_counter_to_activate_long_threshold = LONG_THRESH*long_window
  safety_window = 1.5*avg_counter_to_activate_long_threshold
  long_thresh_met = 0

  long_start_selection = 0
  while True:
    data = stream.read(CHUNK, exception_on_overflow=False)
    recorded_frames.append(data)
    frame_data = struct.unpack(str(CHUNK) + 'h', data)
    frame_data = np.array(frame_data)
    sum_abs_frame = np.sum(np.abs(frame_data))
    quantized_history.append(0 if sum_abs_frame < j_lookup[Jn]['NOISE_FLOOR'] else 1)
    current_levels[Jn] = sum_abs_frame

    counter = len(recorded_frames)
    current_lens[Jn] = counter
    if counter >= long_window:
      long_movavg = sum(quantized_history[counter-long_window:counter])/long_window
      if long_movavg >= LONG_THRESH and long_thresh_met != 1:
        long_start_selection = int(max(counter - safety_window, 0))
        long_thresh_met = 1
      if long_movavg < LONG_THRESH and long_thresh_met == 1:
        long_end = int(counter)
        long_thresh_met = 2
        save_to_disk(recorded_frames[long_start_selection:long_end], audio, Jn)

    if counter > MAX_LOOKBACK_PERIOD: # don't keep endless audio history to avoid excessive memory usage
      del recorded_frames[0]
      del quantized_history[0]
      long_start_selection = max(0, long_start_selection - 1) # since you deleted first element, the recording start index is now one less

核心原因 #

你的推测完全正确：Python列表的del list[0]是**O(n)**复杂度操作，删除头部元素后所有后续元素都要向前移动。当数据积累到4小时量级时，单次del会占用大量CPU时间，导致主循环无法及时调用stream.read()，触发exception_on_overflow=False时的静默丢帧，最终表现为音频频谱的垂直尖峰失真。

解决思路 #

1. 改用环形缓冲区替代普通列表 #

彻底避免头部删除操作，用固定大小的环形缓冲区循环复用存储空间：

• 初始化两个长度为MAX_LOOKBACK_PERIOD的数组，用指针标记当前写入位置
• 每次写入时覆盖旧数据，无需删除元素
• 计算移动平均时，通过指针偏移确定窗口内的有效数据
  示例代码片段：

# 初始化环形缓冲区
recorded_frames = [b''] * MAX_LOOKBACK_PERIOD
quantized_history = [0] * MAX_LOOKBACK_PERIOD
write_ptr = 0
rolling_sum = 0  # 配合滚动和优化移动平均计算

while True:
    data = stream.read(CHUNK, exception_on_overflow=False)
    # 覆盖旧数据
    old_quantized = quantized_history[write_ptr]
    recorded_frames[write_ptr] = data
    
    # 处理当前帧
    frame_data = struct.unpack(str(CHUNK) + 'h', data)
    frame_data = np.array(frame_data)
    sum_abs_frame = np.sum(np.abs(frame_data))
    new_quantized = 0 if sum_abs_frame < j_lookup[Jn]['NOISE_FLOOR'] else 1
    quantized_history[write_ptr] = new_quantized
    
    # 更新滚动和
    rolling_sum += new_quantized - old_quantized
    current_levels[Jn] = sum_abs_frame
    
    # 计算当前有效数据长度
    counter = write_ptr + 1 if write_ptr < MAX_LOOKBACK_PERIOD else MAX_LOOKBACK_PERIOD
    current_lens[Jn] = counter
    
    if counter >= long_window:
        # 计算移动平均（滚动和直接用，O(1)）
        long_movavg = rolling_sum / long_window
        
        if long_movavg >= LONG_THRESH and long_thresh_met != 1:
            # 计算环形缓冲区中的起始位置
            long_start_selection = (write_ptr - safety_window) % MAX_LOOKBACK_PERIOD
            long_thresh_met = 1
        if long_movavg < LONG_THRESH and long_thresh_met == 1:
            # 提取需要保存的片段，处理环形缓冲区的边界情况
            if long_start_selection <= write_ptr:
                save_frames = recorded_frames[long_start_selection:write_ptr+1]
            else:
                save_frames = recorded_frames[long_start_selection:] + recorded_frames[:write_ptr+1]
            save_to_disk(save_frames, audio, Jn)
            long_thresh_met = 2
    
    # 移动写入指针，循环复用缓冲区
    write_ptr = (write_ptr + 1) % MAX_LOOKBACK_PERIOD

2. 批量清理减少删除频率 #

如果坚持使用普通列表，不要每次循环删1个元素，改为批量删除：

• 设置批量大小（如BATCH_SIZE=100），当数据量超过阈值+批量大小时，一次性删除前N个元素
• 大幅降低del操作的频率，减少单次CPU占用
  示例调整：

BATCH_SIZE = 100  # 根据实际情况调整
if counter > MAX_LOOKBACK_PERIOD + BATCH_SIZE:
    del recorded_frames[:BATCH_SIZE]
    del quantized_history[:BATCH_SIZE]
    long_start_selection = max(0, long_start_selection - BATCH_SIZE)

3. 优化移动平均计算 #

当前sum(quantized_history[counter-long_window:counter])是O(n)操作，叠加del耗时会进一步拖慢循环：

• 维护滚动和变量，新增元素时加，移除旧元素时减，计算平均只需O(1)时间
• 结合环形缓冲区使用时效果最佳，单独使用也能降低主循环负载

4. 异步处理耗时任务 #

把内存清理、音频保存等耗时操作放到独立线程中执行，避免阻塞主循环的音频读取：

• 主循环只负责读音频、实时计算，将任务放入线程安全队列
• 后台线程从队列取任务异步执行，注意用threading.Lock保护共享数据

内容的提问来源于stack exchange，提问作者EllipticalInitial