使用Whisper AI转录音频时遭遇TypeError错误求助

问题背景 #

已配置Python 3.8.16、PyTorch 1.12.1并安装最新版ffmpeg，在Jupyter Notebook运行Whisper转录代码时触发TypeError。

版本验证结果：

3.8.16 | packaged by conda-forge | (default, Feb  1 2023, 15:53:35) [MSC v.1929 64 bit (AMD64)]
1.12.1

运行代码：

import openai
import whisper

openai.api_key = "my open ai key"

audio_file= open(r"C:\Users\Chris\Python scripts\test_audio.mp3", "rb")

model = whisper.load_model("base")
result = model.transcribe(audio_file)

print(result)

报错信息：

TypeError                                 Traceback (most recent call last)
Cell In[28], line 9
      6 audio_file= open(r"C:\Users\Chris\Python scripts\test_audio.mp3", "rb")
      8 model = whisper.load_model("base")
----> 9 result = model.transcribe(audio_file)
     11 print(result)

File ~\anaconda3\lib\site-packages\whisper\transcribe.py:121, in transcribe(model, audio, verbose, temperature, compression_ratio_threshold, logprob_threshold, no_speech_threshold, condition_on_previous_text, initial_prompt, word_timestamps, prepend_punctuations, append_punctuations, **decode_options)
    118     decode_options["fp16"] = False
    120 # Pad 30-seconds of silence to the input audio, for slicing
--> 121 mel = log_mel_spectrogram(audio, padding=N_SAMPLES)
    122 content_frames = mel.shape[-1] - N_FRAMES
    124 if decode_options.get("language", None) is None:

File ~\anaconda3\lib\site-packages\whisper\audio.py:131, in log_mel_spectrogram(audio, n_mels, padding, device)
    129     if isinstance(audio, str):
    130         audio = load_audio(audio)
--> 131     audio = torch.from_numpy(audio)
    133 if device is not None:
    134     audio = audio.to(device)

TypeError: expected np.ndarray (got _io.BufferedReader)

解决方法 #

错误核心原因：model.transcribe()不接受文件对象作为参数，它仅支持音频文件路径字符串或加载完成的numpy数组作为输入。

方法1：直接传入文件路径（推荐） #

删除打开文件的代码，直接将音频路径字符串传入transcribe，同时注意本地Whisper转录无需OpenAI API密钥，可移除相关代码：

import whisper

model = whisper.load_model("base")
# 直接传入音频文件路径
result = model.transcribe(r"C:\Users\Chris\Python scripts\test_audio.mp3")

# 直接输出转录文本更直观
print(result["text"])

方法2：手动加载音频为numpy数组 #

若需先处理音频数据，可使用Whisper自带的load_audio函数将文件加载为numpy数组后传入：

import whisper
from whisper.audio import load_audio

# 加载音频为numpy数组
audio_data = load_audio(r"C:\Users\Chris\Python scripts\test_audio.mp3")

model = whisper.load_model("base")
result = model.transcribe(audio_data)

print(result["text"])

内容的提问来源于stack exchange，提问作者Chris