React+Flask实时语音转文本故障求助：Azure Speech SDK调用异常

一、前端（React）音频采集与Socket传输优化 #

• 强制音频格式匹配Azure要求：必须是16kHz采样率、16位深度、单声道PCM格式，直接传输原始PCM字节，避免浏览器自带的格式转换干扰识别。
• 依赖安装：

  npm install socket.io-client
  

• 核心代码示例：

import { useEffect, useRef } from 'react';
import io from 'socket.io-client';

const SpeechToText = () => {
  const audioContextRef = useRef(null);
  const scriptProcessorRef = useRef(null);
  const socketRef = useRef(null);

  const startRecording = async () => {
    // 请求麦克风权限，指定音频参数
    const stream = await navigator.mediaDevices.getUserMedia({
      audio: { sampleRate: 16000, channelCount: 1, echoCancellation: true }
    });

    // 初始化音频上下文
    audioContextRef.current = new AudioContext({ sampleRate: 16000 });
    const mediaSource = audioContextRef.current.createMediaStreamSource(stream);
    
    // 创建脚本处理器，每帧处理4096个样本（平衡实时性与传输效率）
    scriptProcessorRef.current = audioContextRef.current.createScriptProcessor(4096, 1, 1);
    mediaSource.connect(scriptProcessorRef.current);
    scriptProcessorRef.current.connect(audioContextRef.current.destination);

    // 初始化Socket连接
    socketRef.current = io('http://your-flask-server:5000', { transports: ['websocket'] });
    socketRef.current.on('transcript', (data) => {
      // 更新UI展示实时转录文本
      console.log('实时结果:', data.text);
    });

    // 处理音频数据并发送
    scriptProcessorRef.current.onaudioprocess = (e) => {
      const channelData = e.inputBuffer.getChannelData(0);
      // 将Float32格式转换为16位PCM字节数组
      const pcmData = new Int16Array(channelData.length);
      for (let i = 0; i < channelData.length; i++) {
        const sample = Math.max(-1, Math.min(1, channelData[i]));
        pcmData[i] = sample < 0 ? sample * 0x8000 : sample * 0x7FFF;
      }
      socketRef.current.emit('audio_data', pcmData.buffer);
    };
  };

  const stopRecording = () => {
    if (scriptProcessorRef.current) {
      scriptProcessorRef.current.disconnect();
    }
    if (audioContextRef.current) {
      audioContextRef.current.close();
    }
    if (socketRef.current) {
      socketRef.current.emit('stop_recording');
      socketRef.current.disconnect();
    }
  };

  return (
    <div>
      <button onClick={startRecording}>开始录音</button>
      <button onClick={stopRecording}>停止录音</button>
    </div>
  );
};

export default SpeechToText;

二、后端（Flask）Azure Speech SDK正确配置 #

• 依赖安装：

  pip install flask-socketio azure-cognitiveservices-speech eventlet
  

• 核心代码示例：

from flask import Flask, request
from flask_socketio import SocketIO, emit
import azure.cognitiveservices.speech as speechsdk
import threading
from queue import Queue

app = Flask(__name__)
# 强制使用WebSocket传输，避免轮询导致的延迟
socketio = SocketIO(app, cors_allowed_origins="*", async_mode='eventlet', transports=['websocket'])

# Azure Speech服务配置，替换为你的密钥和区域
speech_config = speechsdk.SpeechConfig(
    subscription="YOUR_AZURE_SPEECH_KEY",
    region="YOUR_AZURE_REGION"
)
# 绑定音频格式，与前端完全一致
audio_format = speechsdk.audio.AudioFormat(
    sample_rate_hz=16000,
    bits_per_sample=16,
    channels=1
)

# 自定义Push流，用队列缓存前端传来的音频数据
class CustomPushStream(speechsdk.audio.PushAudioInputStream):
    def __init__(self, audio_format):
        super().__init__(audio_format)
        self.data_queue = Queue()
        self.is_closed = False

    def write(self, data):
        if not self.is_closed:
            self.data_queue.put(data)

    def close(self):
        self.is_closed = True
        self.data_queue.put(None)

    def read(self, size):
        data_chunk = b''
        while len(data_chunk) < size:
            chunk = self.data_queue.get()
            if chunk is None:
                return data_chunk
            data_chunk += chunk
        return data_chunk[:size]

# 启动实时识别线程
def run_recognition(stream, sid):
    audio_config = speechsdk.audio.AudioConfig(stream=stream)
    # 单用户识别用SpeechRecognizer即可，无需ConversationTranscriber
    recognizer = speechsdk.SpeechRecognizer(
        speech_config=speech_config,
        audio_config=audio_config,
        language="zh-CN"  # 根据需求切换语言，如"en-US"
    )

    # 绑定实时识别事件
    recognizer.recognizing.connect(lambda evt: emit('transcript', {'text': evt.result.text}, room=sid))
    recognizer.recognized.connect(lambda evt: emit('transcript', {'text': evt.result.text}, room=sid))
    recognizer.canceled.connect(lambda evt: print(f"识别错误: {evt.reason} - {evt.error_details}"))

    recognizer.start_continuous_recognition_async()

# 接收前端音频数据
@socketio.on('audio_data')
def handle_audio(data):
    sid = request.sid
    # 为每个连接创建独立的音频流和识别实例
    if 'audio_stream' not in socketio.server.environ[sid]:
        stream = CustomPushStream(audio_format)
        socketio.server.environ[sid]['audio_stream'] = stream
        threading.Thread(target=run_recognition, args=(stream, sid), daemon=True).start()
    stream = socketio.server.environ[sid]['audio_stream']
    stream.write(data)

# 处理停止录音请求
@socketio.on('stop_recording')
def handle_stop():
    sid = request.sid
    if 'audio_stream' in socketio.server.environ[sid]:
        stream = socketio.server.environ[sid]['audio_stream']
        stream.close()
        del socketio.server.environ[sid]['audio_stream']

if __name__ == '__main__':
    socketio.run(app, host='0.0.0.0', port=5000, debug=True)

三、常见问题排查 #

• 音频格式不匹配：用Audacity等工具检查前端传输的PCM数据，确认采样率、声道数、位深度与Azure配置完全一致。
• 无识别输出：查看Azure SDK错误日志，若提示认证失败，检查密钥和区域是否正确；若提示格式不支持，核对音频参数。
• 结果不准确：开启前端回声消除，或在Azure配置中添加噪音抑制参数：

  speech_config.set_property(speechsdk.PropertyId.SpeechServiceConnection_EndSilenceTimeoutMs, "3000")
  speech_config.set_property(speechsdk.PropertyId.SpeechServiceConnection_InitialSilenceTimeoutMs, "1000")
  

• 数据丢失：强制使用WebSocket传输（前端transports: ['websocket']），避免轮询导致的数据包丢失。

内容的提问来源于stack exchange，提问作者Sandeep pradeep