核心问题分析 #

当前代码的卡顿源于播放操作阻塞了音频接收流程，且缺乏缓冲机制应对网络传输的不稳定：每收到一小段音频就立即调用play()，该方法是阻塞式的，会暂停后续的音频chunk接收；同时零散的小片段播放本身也容易产生断续感。

具体优化方法 #

1. 线程分离+队列缓冲（最有效）

将音频的**接收（生产）与播放（消费）**拆分为两个独立线程，用队列存储待播放的音频片段，避免播放阻塞接收流程，同时缓冲一定量的音频应对网络波动。

示例代码：

import queue
import threading
from pydub import AudioSegment
from pydub.playback import play
import openai
import base64
import io

# 初始化音频队列，设置最大长度防止内存溢出
audio_queue = queue.Queue(maxsize=10)
stop_event = threading.Event()

def audio_consumer():
    """音频播放线程：从队列取片段播放"""
    while not stop_event.is_set() or not audio_queue.empty():
        try:
            audio_segment = audio_queue.get(timeout=1)
            play(audio_segment)
            audio_queue.task_done()
        except queue.Empty:
            continue

def main():
    full_audio = AudioSegment.empty()
    client = openai.OpenAI()

    # 启动播放线程
    consumer_thread = threading.Thread(target=audio_consumer)
    consumer_thread.start()

    stream_completion = client.chat.completions.create(
        model="gpt-4o-audio-preview",
        modalities=["text", "audio"],
        audio={"voice": "alloy", "format": "pcm16"},
        messages=[
            {"role": "user", "content": "Can you tell me a funny short story about a pickle?"}
        ],
        stream=True
    )

    for chunk in stream_completion:
        chunk_audio = getattr(chunk.choices[0].delta, 'audio', None)
        if chunk_audio is not None:
            pcm_bytes = base64.b64decode(chunk_audio.get('data', ''))
            if pcm_bytes:
                audio_segment = AudioSegment.from_raw(
                    io.BytesIO(pcm_bytes),
                    sample_width=2,
                    frame_rate=24000,
                    channels=1
                )
                full_audio += audio_segment
                # 将音频片段放入队列，不阻塞接收流程
                try:
                    audio_queue.put(audio_segment, block=False)
                except queue.Full:
                    # 队列满时跳过旧片段，优先处理新数据
                    pass

    # 等待队列中所有音频播放完成
    audio_queue.join()
    stop_event.set()
    consumer_thread.join()

    full_audio.export("assets/audio/full_audio.wav", format="wav")

if __name__ == "__main__":
    main()

2. 增加预缓冲阈值

在开始播放前，先积累一定时长的音频（比如1-2秒），避免因网络延迟导致的断续。可以在接收线程中统计缓冲的音频总时长，达到阈值后再启动播放线程。

示例逻辑：

# 在接收chunk时累计音频时长
buffer_duration = 0
buffer_segments = []
target_buffer = 2000  # 预缓冲2000毫秒（2秒）

for chunk in stream_completion:
    # ... 解析音频片段 ...
    buffer_segments.append(audio_segment)
    buffer_duration += len(audio_segment)
    if buffer_duration >= target_buffer and not consumer_thread.is_alive():
        # 将缓冲的片段全部放入队列，启动播放线程
        for seg in buffer_segments:
            audio_queue.put(seg)
        consumer_thread.start()
        buffer_segments = []  # 清空缓冲列表
    elif consumer_thread.is_alive():
        audio_queue.put(audio_segment)

3. 替换高效的音频播放库

pydub.playback.play()基于简单音频库，对于流式播放的支持有限。可以改用simpleaudio直接播放原始PCM数据，减少音频格式转换的开销；或用pygame实现低延迟的流式播放。

示例（用simpleaudio播放原始PCM）：

import simpleaudio as sa

# 在播放线程中直接播放PCM数据
def audio_consumer():
    while not stop_event.is_set() or not audio_queue.empty():
        try:
            pcm_bytes = audio_queue.get(timeout=1)
            # 直接创建音频对象播放
            play_obj = sa.play_buffer(
                pcm_bytes,
                num_channels=1,
                bytes_per_sample=2,
                sample_rate=24000
            )
            play_obj.wait_done()
            audio_queue.task_done()
        except queue.Empty:
            continue

# 接收线程中直接存入原始pcm_bytes，无需转换为AudioSegment
pcm_bytes = base64.b64decode(chunk_audio.get('data', ''))
if pcm_bytes:
    audio_queue.put(pcm_bytes)

4. 合并小chunk再播放

将多个连续的小音频chunk合并为一个较大的片段后再播放，减少播放的启动/停止次数，降低卡顿感。

示例逻辑：

current_chunk = b""
min_chunk_size = 24000 * 2 * 0.5  # 合并至少0.5秒的音频（采样率24000*2字节/秒*0.5）

for chunk in stream_completion:
    # ... 解析pcm_bytes ...
    current_chunk += pcm_bytes
    if len(current_chunk) >= min_chunk_size:
        audio_segment = AudioSegment.from_raw(
            io.BytesIO(current_chunk),
            sample_width=2,
            frame_rate=24000,
            channels=1
        )
        audio_queue.put(audio_segment)
        current_chunk = b""
# 处理最后剩余的小chunk
if current_chunk:
    audio_segment = AudioSegment.from_raw(io.BytesIO(current_chunk), sample_width=2, frame_rate=24000, channels=1)
    audio_queue.put(audio_segment)

内容的提问来源于stack exchange，提问作者Seth Brock