在RunPod上对比WhisperX与Faster-Whisper：速度、准确率及优化

最近我在RunPod的服务器上对比了WhisperX和Faster-Whisper的性能，用到的核心代码片段如下：

WhisperX 实现代码 #

import time
import whisperx

model = whisperx.load_model(
    "large-v3", "cuda"
)

def run_whisperx_job(job):
    start_time = time.time()

    job_input = job['input']
    url = job_input.get('url', "")

    print(f" Loading audio from {url}...")
    audio = whisperx.load_audio(url)
    print("✅ Audio loaded")

    print("Transcribing...")
    result = model.transcribe(audio, batch_size=16)

    end_time = time.time()
    time_s = (end_time - start_time)
    print(f" Transcription done: {time_s:.2f} s")
    # print(result)

    # 为了便于迁移，我们沿用了RunPod官方Faster-Whisper的输出格式
    output = {
        'detected_language' : result['language'],
        'segments' : result['segments']
    }

    return output

Faster-Whisper 实现代码 #

import time
import os
from faster_whisper import WhisperModel
import rp_cleanup

# 加载Faster-Whisper模型
model = WhisperModel("large-v3", device="cuda", compute_type="float16")

def run_faster_whisper_job(job):
    start_time = time.time()
    
    job_input = job['input']
    url = job_input.get('url', "")

    print(f" Downloading audio from {url}...")
    audio_path = download_files_from_urls(job['id'], [url])[0]
    print("✅ Audio downloaded")
    
    print("Transcribing...")
    segments, info = model.transcribe(audio_path, beam_size=5)
    
    output_segments = []
    for segment in segments:
        output_segments.append({
            "start": segment.start,
            "end": segment.end,
            "text": segment.text
        })
    
    end_time = time.time()
    time_s = (end_time - start_time)
    print(f" Transcription done: {time_s:.2f} s")
    
    output = {
        'detected_language': info.language,
        'segments': output_segments
    }
    
    # ✅ 转录完成后安全删除文件
    try:
        if os.path.exists(audio_path):
            os.remove(audio_path)  # 使用os.remove()
            print(f"️ Deleted {audio_path}")
        else:
            print("⚠️ File not found, skipping deletion")
    except Exception as e:
        print(f"❌ Error deleting file: {e}")

    rp_cleanup.clean(['input_objects'])

    return output

整体测试结论 #

• WhisperX的处理速度明显快于Faster-Whisper
• WhisperX可以顺利处理3小时的长音频，而Faster-Whisper会出现未知运行时错误。我猜测这是因为Faster-Whisper需要更多的GPU内存资源才能完成这类长任务

准确率对比结果 #

• WhisperX的准确率不如Faster-Whisper
• WhisperX的转录结果中遗漏的词语比Faster-Whisper更多

优化疑问 #

我现在想知道，在WhisperX里可以调整哪些参数或者做哪些微调，能够：

• 提升转录准确率
• 减少词语遗漏的情况
• 同时不会大幅增加处理时间

备注：内容来源于stack exchange，提问作者Cheok Yan Cheng