Azure Speech SDK发音评估始终返回AccuracyScore=0求助

问题描述 #

我尝试在代码中使用Azure Speech SDK添加发音评估功能，代码实现如下：

public static AudioConfig CreateAudioConfigFromBytes(byte[] audioBytes)
{
    var audioStream = new MemoryStream(audioBytes);

    var pushStream = AudioInputStream.CreatePushStream();

    pushStream.Write(audioBytes);
    pushStream.Close();

    var audioConfig = AudioConfig.FromStreamInput(pushStream);
    return audioConfig;
}


public async Task<float> AssessPronunciation(byte[] audioBytes, string referenceText)
{
    string subscriptionKey = Environment.GetEnvironmentVariable("STT_API_KEY");

    var pronunciationAssessmentConfig = new PronunciationAssessmentConfig(
        referenceText: referenceText,
        gradingSystem: GradingSystem.HundredMark,
        granularity: Granularity.Phoneme,
        enableMiscue: false);

    var audioConfig = CreateAudioConfigFromBytes(audioBytes);
    var speechConfig = SpeechConfig.FromSubscription(subscriptionKey, "eastus");

    using (var speechRecognizer = new SpeechRecognizer(speechConfig, audioConfig))
    {
        pronunciationAssessmentConfig.ApplyTo(speechRecognizer);
        var speechRecognitionResult = await speechRecognizer.RecognizeOnceAsync();

        // The pronunciation assessment result as a Speech SDK object
        var pronunciationAssessmentResult = PronunciationAssessmentResult.FromResult(speechRecognitionResult);

        // The pronunciation assessment result as a JSON string
        var pronunciationAssessmentResultJson = speechRecognitionResult.Properties.GetProperty(PropertyId.SpeechServiceResponse_JsonResult);
    }
}

测试时，无论发音正确与否，始终得到AccuracyScore = 0。例如测试单词“school”，正确发音和错误发音（说“pajama”）两次结果均为0。但在Azure门户中测试功能正常：正确发音“school”得分超90，错误发音“pajama”得分为2。

问题原因与修复方案 #

1. PushStream被提前关闭 #

CreateAudioConfigFromBytes方法中，写入音频字节后立刻调用pushStream.Close()，导致SpeechRecognizer尝试读取流时已无法获取数据，识别失败后返回的得分自然为0。PushStream必须保持打开状态，直到SpeechRecognizer完成识别。

修复后的CreateAudioConfigFromBytes：

public static AudioConfig CreateAudioConfigFromBytes(byte[] audioBytes)
{
    var pushStream = AudioInputStream.CreatePushStream();
    pushStream.Write(audioBytes);
    // 不要手动关闭流，SpeechRecognizer会在识别完成后处理
    return AudioConfig.FromStreamInput(pushStream);
}

2. 未检查识别结果状态 #

代码中没有判断识别是否成功，如果流提前关闭或音频格式错误，speechRecognitionResult.Reason会是ResultReason.NoMatch或其他失败状态，此时生成的发音评估结果得分就是0，必须添加状态校验。

3. 方法未返回得分 #

原方法声明返回float，但代码末尾没有返回AccuracyScore，这是语法错误，同时也导致无法获取正确结果。

4. 音频格式不匹配（潜在问题） #

Azure Speech SDK要求音频为16kHz采样率、16位单声道PCM（WAV格式），如果你的音频字节不符合这个格式，识别会失败。可以显式指定音频格式避免问题：

public static AudioConfig CreateAudioConfigFromBytes(byte[] audioBytes)
{
    // 指定符合要求的音频格式
    var audioFormat = AudioFormat.GetWaveFormatPCM(16000, 16, 1);
    var pushStream = AudioInputStream.CreatePushStream(audioFormat);
    pushStream.Write(audioBytes);
    return AudioConfig.FromStreamInput(pushStream);
}

修复后的完整代码 #

public static AudioConfig CreateAudioConfigFromBytes(byte[] audioBytes)
{
    var audioFormat = AudioFormat.GetWaveFormatPCM(16000, 16, 1);
    var pushStream = AudioInputStream.CreatePushStream(audioFormat);
    pushStream.Write(audioBytes);
    return AudioConfig.FromStreamInput(pushStream);
}

public async Task<float> AssessPronunciation(byte[] audioBytes, string referenceText)
{
    string subscriptionKey = Environment.GetEnvironmentVariable("STT_API_KEY");
    var pronunciationAssessmentConfig = new PronunciationAssessmentConfig(
        referenceText: referenceText,
        gradingSystem: GradingSystem.HundredMark,
        granularity: Granularity.Phoneme,
        enableMiscue: false);

    var audioConfig = CreateAudioConfigFromBytes(audioBytes);
    var speechConfig = SpeechConfig.FromSubscription(subscriptionKey, "eastus");

    using (var speechRecognizer = new SpeechRecognizer(speechConfig, audioConfig))
    {
        pronunciationAssessmentConfig.ApplyTo(speechRecognizer);
        var speechRecognitionResult = await speechRecognizer.RecognizeOnceAsync();

        // 检查识别是否成功
        if (speechRecognitionResult.Reason != ResultReason.RecognizedSpeech)
        {
            // 可根据实际需求处理失败场景，比如日志记录或抛出异常
            Console.WriteLine($"识别失败: {speechRecognitionResult.Reason}");
            return 0;
        }

        var pronunciationAssessmentResult = PronunciationAssessmentResult.FromResult(speechRecognitionResult);
        // 返回正确的发音准确度得分
        return pronunciationAssessmentResult.AccuracyScore;
    }
}

内容的提问来源于stack exchange，提问作者tzviya