Azure TTS流式传输音频失真问题排查求助

数据老陈

2026-6-17

Azure TTS流式传输音频后期失真问题排查与解决

我用JavaScript的microsoft-cognitiveservices-speech-sdk中SpeechSynthesizer生成Audio16Khz32KBitRateMonoMp3格式音频，通过Express流式传输到React前端时，前几句音频正常，但后面的语音严重失真。

发送音频的代码

synthesizer.synthesizing = function (s, e) {
  currentAudioChunk = {
    audio: Buffer.from(e.result.audioData),
    offset: e.result.audioDuration / 10000, // 转换为毫秒
  };

  sendEvent("audioData", {
    audio: currentAudioChunk.audio.toString("base64"),
    // 目前audioOffset数据为null，暂用占位符
    audioOffset: "0",
  });

  currentAudioChunk = null;
};

音频数据对比

正常时的base64片段：

//NIxCElU/5IAY+IAYJ/2sMoSxHk0RH9BVzA0GTFmDQIAQ4Xv8iBn5oEqAUwtPDlBnyDh9f+ukXFoLdRAhSgoYRmQwQAIIJ0DVH/+eez/FwFkR+DcgYghGI/ImQQ0kU/////LRuTiDFw8fLRmk9Bv////v/+hTTN0Cuzk4TZ8qGCFf6UeMhhgB8BjVMd/t5V//NIxA4g2tqkAc9YAD7MqNv/AVisORkvL3sJA7Fje5e+5N7r/ZNxXO9991vl98n9A+bqGiY3myJPJAOEDjHGJfRMWtAsfGoO9ejd0GhoOy32zfEvfHvZL4vfvTvfV5xO2TeymMriv6//j+v5/j2MNzc+w4oEElx7v6u/2t/UcW0Tg+oDGEGSAs30kiaCvEsa//NIxA0f6tbFtGsQsB/wSwiDZ+tZaEobTf5rXCqf/XqycoAzWXtvSKlDpriVZX/mmfM+YFhrAo1ookTMDgiGJRKHii37Q6Eh+fFkyMUXF4c2/fTr+0rpvsyXQevBDueX8PPz/PPX88VXXxySFgZQJQqJRW71f/Ss+4XRpRipAO1XGSKjcylABX/VcJmGEBBG

失真后的base64片段（含大量重复无效字符）：

//NIxHwAAANIAAAAAFVVVVVVVVVMQU1FMy4xMDBVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV//NIxHwAAANIAAAAAFVVVVVVVVVMQU1FMy4xMDBVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV//NIxHwAAANIAAAAAFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV

这些重复的V字符是从Azure接收的原始数据中就存在的，不是字符串转换导致的，直接删除会损坏音频数据。

解决方法

1. 切换音频格式，降低流式解析压力

MP3是带帧结构的压缩格式，流式传输时容易因不完整帧导致失真。可以先使用Audio16Khz16BitMonoPcm格式（无压缩的PCM），传输到前端后再转码为MP3，避免帧不完整的问题。修改合成器初始化代码：

const speechConfig = SpeechConfig.fromSubscription(subscriptionKey, region);
speechConfig.speechSynthesisOutputFormat = SpeechSynthesisOutputFormat.Audio16Khz16BitMonoPcm;

2. 缓存并校验完整MP3帧

如果坚持使用MP3格式，需要缓存音频块，直到收集到完整的MP3帧再发送。MP3帧以同步字（通常是0xFFFB）开头，我们可以通过检测同步字来分割完整帧：

// 缓存未完成的MP3帧
let incompleteFrame = Buffer.from([]);

synthesizer.synthesizing = function (s, e) {
  let audioChunk = Buffer.from(e.result.audioData);
  // 合并缓存的不完整帧和当前块
  let combined = Buffer.concat([incompleteFrame, audioChunk]);
  
  // 查找最后一个完整帧的起始位置
  let syncWord = Buffer.from([0xFF, 0xFB]);
  let lastFrameStart = combined.lastIndexOf(syncWord);
  
  if (lastFrameStart > 0) {
    // 提取完整帧发送
    let completeFrames = combined.slice(0, lastFrameStart);
    sendEvent("audioData", {
      audio: completeFrames.toString("base64"),
      audioOffset: (e.result.audioDuration / 10000).toString()
    });
    // 缓存剩余的不完整部分
    incompleteFrame = combined.slice(lastFrameStart);
  } else {
    // 没有找到完整帧，全部缓存
    incompleteFrame = combined;
  }
};

// 合成结束时发送剩余缓存的帧
synthesizer.synthesisCompleted = function (s, e) {
  if (incompleteFrame.length > 0) {
    sendEvent("audioData", {
      audio: incompleteFrame.toString("base64"),
      audioOffset: "0"
    });
    incompleteFrame = Buffer.from([]);
  }
};

3. 修正音频偏移量

当前代码中audioOffset用固定值"0"，会导致前端拼接音频时时序混乱。应该使用e.result.audioDuration计算正确的偏移量，让前端按顺序拼接音频块：

sendEvent("audioData", {
  audio: currentAudioChunk.audio.toString("base64"),
  audioOffset: (e.result.audioDuration / 10000).toString()
});

4. 检查网络传输稳定性

流式传输中丢包或数据截断也会导致失真，可以在传输时添加简单的校验（比如每个块附带长度信息），前端收到后验证长度是否匹配，确保数据完整。

内容的提问来源于stack exchange，提问作者Janet Gilbert