Next.js集成OpenAI语音转文字在Safari识别异常求助

我做了一个基于Next.js集成OpenAI语音转文字的项目，在Chrome浏览器中运行正常，转写内容准确，但在Safari浏览器中返回的转写文本完全错误。相关代码如下：

useRecordVoice.js #

"use client";
import { useEffect, useState, useRef } from "react";
import { blobToBase64 } from "@/utils/blobToBase64";
import { createMediaStream } from "@/utils/createMediaStream";

export const useRecordVoice = () => {
  const [text, setText] = useState("");
  const [mediaRecorder, setMediaRecorder] = useState(null);
  const [recording, setRecording] = useState(false);
  const isRecording = useRef(false);
  const chunks = useRef([]);

  const startRecording = () => {
    if (mediaRecorder) {
      isRecording.current = true;
      mediaRecorder.start();
      setRecording(true);
    }
  };

  const stopRecording = () => {
    if (mediaRecorder) {
      isRecording.current = false;
      mediaRecorder.stop();
      setRecording(false);
    }
  };

  const getText = async (base64data) => {
    try {
      const response = await fetch("/api/speechToText", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          audio: base64data,
        }),
      }).then((res) => res.json());
      const { text } = response;
      setText(text);
    } catch (error) {
      console.log(error);
    }
  };

  const initialMediaRecorder = (stream) => {
    const mediaRecorder = new MediaRecorder(stream);

    mediaRecorder.onstart = () => {
      createMediaStream(stream);
      chunks.current = [];
    };

    mediaRecorder.ondataavailable = (ev) => {
      chunks.current.push(ev.data);
    };

    mediaRecorder.onstop = () => {
      const audioBlob = new Blob(chunks.current, { type: "audio/wav" });
      blobToBase64(audioBlob, getText);
    };

    setMediaRecorder(mediaRecorder);
  };

  useEffect(() => {
    if (typeof window !== "undefined") {
      navigator.mediaDevices
        .getUserMedia({ audio: true })
        .then(initialMediaRecorder);
    }
  }, []);

  return { recording, startRecording, stopRecording, text };
};

API speech-to-text #

import { NextResponse } from "next/server";
import fs from "fs";
import * as dotenv from "dotenv";
import OpenAI from "openai";
import { env } from "../../config/env";

dotenv.config();

const openai = new OpenAI({
  apiKey: env.OPENAI_API_KEY,
});

export async function POST(req) {
  const body = await req.json();
  const base64Audio = body.audio;
  const audio = Buffer.from(base64Audio, "base64");
  const filePath = "tmp/input.wav";

  try {
    fs.writeFileSync(filePath, audio);
    const readStream = fs.createReadStream(filePath);
    const data = await openai.audio.transcriptions.create({
      file: readStream,
      model: "whisper-1",
    });
    // Remove the file after use
    fs.unlinkSync(filePath);

    return NextResponse.json(data);
  } catch (error) {
    console.error("Error processing audio:", error);
    return NextResponse.error();
  }
}

核心原因：浏览器间MediaRecorder的音频格式差异 #

1. 默认编码不匹配：Chrome的MediaRecorder默认输出audio/webm（Opus编码），Safari默认输出audio/mp4（AAC编码）。你在创建Blob时强制指定type: "audio/wav"，但这只是修改了Blob的MIME类型标记，并没有真正将音频编码转换为WAV格式，导致传给Whisper的是伪装成WAV的压缩音频，Whisper无法正确解析，从而输出错误内容。
2. Whisper的格式要求：OpenAI Whisper对输入音频的要求是PCM编码的无压缩WAV格式（16位、单声道、16kHz采样率最佳），而Safari录制的AAC压缩音频不符合该要求，即使修改MIME类型也无法被正确识别。
3. Safari对WAV的支持限制：Safari无法直接录制为WAV格式，强制指定audio/wav作为MediaRecorder的输出类型会导致录制失败或生成无效文件。

修复方案 #

1. 指定兼容的录制格式
   初始化MediaRecorder时，优先选择跨浏览器支持的编码格式，避免强制指定WAV：

   const initialMediaRecorder = (stream) => {
     // 优先选WebM Opus（兼容Chrome和Safari 14.1+），降级到AAC
     const mimeType = MediaRecorder.isTypeSupported('audio/webm;codecs=opus') 
       ? 'audio/webm;codecs=opus' 
       : 'audio/mp4;codecs=aac';
     const mediaRecorder = new MediaRecorder(stream, { mimeType });
   
     // 剩余逻辑不变
   };
   

2. 后端强制转换为标准WAV格式
   在API中添加音频格式转换步骤，将上传的音频转为Whisper适配的PCM WAV，可使用ffmpeg工具处理：

   // 需先安装fluent-ffmpeg和ffmpeg依赖
   import ffmpeg from 'fluent-ffmpeg';
   import { promisify } from 'util';
   const exec = promisify(require('child_process').exec);
   
   // 替换POST函数中的文件处理部分
   await exec(`ffmpeg -i ${filePath} -acodec pcm_s16le -ar 16000 -ac 1 tmp/converted.wav`);
   const readStream = fs.createReadStream('tmp/converted.wav');
   // 后续传给Whisper...
   

3. 验证音频有效性
   前端可添加录制后音频预览功能，确认Safari下录制的音频能正常播放；后端可在接收文件后检查实际编码格式，避免无效文件传入Whisper。

内容的提问来源于stack exchange，提问作者Zahar