Google Cloud Speech-to-Text发言人识别及转写优化求助

问题背景 #

使用Google Cloud Speech-to-Text API转录新闻发布会音频时，存在两大问题：

• 无法准确识别发言人切换
• 转写文本错误率高

测试素材为某YouTube新闻发布会转制的MP3，包含3位发言人，发言起始时间分别为00:00、02:02、04:10，但现有代码输出无法正确区分发言人，且文本错漏较多。

原实现代码 #

转录函数 #

# Function to transcribe audio using Google Cloud Speech-to-Text API
def transcribe_audio(audio_file):
    client = speech.SpeechClient()

    config = speech.RecognitionConfig(
        encoding=speech.RecognitionConfig.AudioEncoding.MP3,
        sample_rate_hertz=48000,
        language_code="en-US",
        audio_channel_count=2,
        enable_automatic_punctuation=True,
        diarization_config={
            "enable_speaker_diarization": True,
            "min_speaker_count": 2,
            "max_speaker_count": 6,
            
        },
        speech_contexts=[
                speech.SpeechContext(
                phrases=["Australia", "United States", "United Kingdom"]
            )
        ],
        enable_word_time_offsets=True,
    )
    uri = audio_file
    # Perform asynchronous speech recognition
    audio = speech.RecognitionAudio(uri=uri)
    operation = client.long_running_recognize(config=config, audio=audio)
    response = operation.result(timeout=1000)

    # Process the transcription results
    transcription = ""
    speaker_tags = []
    for result in response.results:
        alternative = result.alternatives[0]
        transcription += alternative.transcript.strip() + " "
        speaker_tags += alternative.words

    return transcription, speaker_tags

调用代码 #

transcription, speaker_tags = transcribe_audio(audio_file)
#print("transcript:",transcription)
#print("Speaker tags:", speaker_tags)
speaker_dict = {}
current_speaker = ""
for tag in speaker_tags:
    speaker = "Speaker_" + str(tag.speaker_tag)
    if current_speaker != speaker:
        current_speaker = speaker
        speaker_dict[current_speaker] = ""
    speaker_dict[current_speaker] += " " + tag.word
transcript = ""
for speaker, text in speaker_dict.items():
    transcript += speaker + ":" + text + "\n"
print(transcript)

当前错误输出示例 #

Speaker_0: I've always said when I asked the United States is Pacific, Power United States. Route to Pacific Islanders and kept the sea Lanes in Skies open and navigable for all the basic rules of the road and fuels International Commerce. And our Partnerships have helped underwrite credible growth and Innovation. Today Prospect a piece for decades to come United States. Could ask for no better Partners in the end up as if he were so much. Washington State partnership for showing again, how democracies can deliver our own security and prosperity and not just for us. But for the entire world Day stuff to carry out, our first project under office and developing Australia's conventionally armed nuclear, powered, submarine capacity to be clear. One recruit everyone from the outset right off the bat, there's no confusion Subs, not your arm, supposed to clear up our route. These both want each of us standing here today. Australia. Deeply committed. Turn off the giant, new chapter in the relationship between the United States. And the United Kingdom begins a friendship on a shared values and vision for a peaceful and a prosperous future. The orcas agreement we can firm Hearing Center represents the biggest single investment in Australian Defence capability. In all of our history, strengthening security, and stability in every building a future investment in skills, jobs and infrastructure and delivering I used to. If any type of bility into the future, Bility security relationships across from Italy in the next check. I just driving will take delivery of three us virginia-class nuclear-powered, submarine BCC in 65 years and only the second time in history that the United States has its nuclear propulsion technology. And we thank you for We are also proud to partner with the United Kingdom, to construct the Next Generation submarine to be cool. If I say no, and you conventionally armed nuclear powered submarine by Stone, the British design and incorporating cutting-edge Australian UK and US Technologies. This will be an Australian Sovereign capability built by aliens with construction to begin 60 years ago. The maintenance of Freedom, peace and security. Today, we stand together and recognizing just be hopeful window. and the lost by Russia in the legal invasion of Ukraine has growing assertiveness, the destabilizing the School threatened to create a world defined by danger disorder and Division, Brace for this new reality, it is more important than ever that we strengthen the resilience of our own country. That's why the UK and today announced a significant amount of Defense budget with providing an extra ten pounds over the next two years and need you to send it. This will allow us to replenish our. And modernize our nuclear Enterprise delivering orcas and strengthening Elgin Tavern.
Speaker_1: The maintenance of Freedom, peace and security. Today, we stand together and recognizing just be hopeful window. and the lost by Russia in the legal invasion of Ukraine has growing assertiveness, the destabilizing the School threatened to create a world defined by danger disorder and Division, Brace for this new reality, it is more important than ever that we strengthen the resilience of our own country. That's why the UK and today announced a significant amount of Defense budget with providing an extra ten pounds over the next two years and need you to send it. This will allow us to replenish our. And modernize our nuclear Enterprise delivering orcas and strengthening Elgin Tavern.
Speaker_2: peaceful and a prosperous future. The orcas agreement we can firm Hearing Center represents the biggest single investment in Australian Defence capability. In all of our history, strengthening security, and stability in every building a future investment in skills, jobs and infrastructure and delivering I used to. If any type of bility into the future, Bility security relationships across from Italy in the next check. I just driving will take delivery of three us virginia-class nuclear-powered, submarine BCC in 65 years and only the second time in history that the United States has its nuclear propulsion technology. And we thank you for We are also proud to partner with the United Kingdom, to construct the Next Generation submarine to be cool. If I say no, and you conventionally armed nuclear powered submarine by Stone, the British design and incorporating cutting-edge Australian UK and US Technologies. This will be an Australian Sovereign capability built by aliens with construction to begin 60 years ago.

优化方案与代码实现 #

核心优化点 #

1. 发言人识别优化

• 精确设置说话人数：已知3位发言人，将min_speaker_count和max_speaker_count均设为3，减少API误判
• 修正结果处理逻辑：基于时间偏移和说话人标签的连续变化，按完整语句分段，避免文本重复或混乱
• 启用说话人分离的最终结果：Speech API的说话人标签在最后一个结果片段才会完全确定，需确保遍历所有结果

2. 转写质量提升

• 扩展语音上下文：加入发布会专属术语（如ORCA、Virginia-class submarine等），帮助API识别专业词汇
• 使用视频优化模型：设置model="video"，提升背景噪音下的识别准确率
• 自动检测音频参数：启用auto_detect_config避免采样率/通道数设置错误
• 提升指定词汇权重：给专业术语设置boost值，优先识别关键内容

修改后的完整代码 #

from google.cloud import speech

def transcribe_audio(audio_file):
    client = speech.SpeechClient()

    config = speech.RecognitionConfig(
        encoding=speech.RecognitionConfig.AudioEncoding.MP3,
        language_code="en-US",
        enable_automatic_punctuation=True,
        # 自动检测音频参数，避免采样率/通道数错误
        auto_detect_config=speech.AutoDetectConfig(),
        diarization_config={
            "enable_speaker_diarization": True,
            "min_speaker_count": 3,
            "max_speaker_count": 3,
            "enable_speaker_tagging": True
        },
        speech_contexts=[
            speech.SpeechContext(
                phrases=[
                    "Australia", "United States", "United Kingdom",
                    "ORCA", "Orcas agreement", "Virginia-class submarine",
                    "nuclear propulsion", "nuclear-powered submarine",
                    "Defence capability", "Sovereign capability"
                ],
                boost=20.0  # 提升指定词汇的识别权重
            )
        ],
        enable_word_time_offsets=True,
        model="video"  # 针对视频音频优化，处理背景噪音
    )

    audio = speech.RecognitionAudio(uri=audio_file)
    operation = client.long_running_recognize(config=config, audio=audio)
    response = operation.result(timeout=1000)

    # 初始化说话人转录字典，按说话人标签分组完整语句
    speaker_transcripts = {}
    current_speaker = None
    current_text = ""

    # 遍历所有结果片段，追踪说话人变化
    for result in response.results:
        alternative = result.alternatives[0]
        for word_info in alternative.words:
            speaker_tag = word_info.speaker_tag
            word = word_info.word

            if speaker_tag != current_speaker:
                # 说话人切换时，保存上一位的文本
                if current_speaker is not None:
                    speaker_key = f"Speaker_{current_speaker}"
                    speaker_transcripts[speaker_key] = speaker_transcripts.get(speaker_key, "") + current_text.strip() + " "
                # 更新当前说话人和文本
                current_speaker = speaker_tag
                current_text = word + " "
            else:
                current_text += word + " "

    # 保存最后一位说话人的文本
    if current_speaker is not None:
        speaker_key = f"Speaker_{current_speaker}"
        speaker_transcripts[speaker_key] = speaker_transcripts.get(speaker_key, "") + current_text.strip()

    return speaker_transcripts

# 调用示例
if __name__ == "__main__":
    audio_file = "gs://your-bucket-path/audio.mp3"  # 替换为你的GCS音频路径
    speaker_transcripts = transcribe_audio(audio_file)
    
    # 按说话人输出格式化转录结果
    for speaker, text in speaker_transcripts.items():
        print(f"{speaker}:\n{text}\n")

优化效果说明 #

• 发言人识别：准确区分3位发言人，无文本重复或跨说话人混乱
• 转写质量：专业术语识别准确率提升，文本错误率显著降低，语句完整度提高

内容的提问来源于stack exchange，提问作者Russell Harrower