使用PyAV创建AudioFrame报错：Unexpected numpy array shape (4800,2)求助

尝试使用PyAV创建音频帧时遇到报错，以下是自定义音频类代码：

class CustomAudioTrack(MediaStreamTrack):
    kind = "audio"

    def __init__(self, rate=48000, channels=2):
        super().__init__()
        self.rate = rate
        self.channels = channels
        self._timestamp = 0

    async def recv(self):
        # 录制0.1秒音频的帧数
        frames = int(self.rate / 10)
        data = sd.rec(frames, samplerate=self.rate,
                      channels=self.channels, dtype=np.int16)
        sd.wait()  # 等待录制完成

        # 更新时间戳
        self._timestamp += frames
        pts = self._timestamp
        time_base = fractions.Fraction(1, self.rate)

        # 创建音频帧
        audio_frame = av.AudioFrame.from_ndarray(
            data, format='s16', layout='stereo')
        audio_frame.sample_rate = self.rate
        audio_frame.pts = pts
        audio_frame.time_base = time_base

        return audio_frame

报错信息：

Traceback (most recent call last):
File "C:\Users\poppppp\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiortc\rtcrtpsender.py", line 328, in _run_rtp
enc_frame = await self._next_encoded_frame(codec)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\poppppp\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiortc\rtcrtpsender.py", line 270, in _next_encoded_frame
data = await self.__track.recv()
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\documents\project\pythonProject\webrtc_tutorial\sadda_python\offerer.py", line 44, in recv
audio_frame = av.AudioFrame.from_ndarray(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "av\audio\frame.pyx", line 125, in av.audio.frame.AudioFrame.from_ndarray
File "av\utils.pyx", line 79, in av.utils.check_ndarray_shape
ValueError: Unexpected numpy array shape (4800, 2)

解决方法 #

PyAV的AudioFrame.from_ndarray要求立体声数据的形状是**(通道数, 采样数)，而当前从sd.rec得到的是(采样数, 通道数)**的形状（即(4800,2)），所以需要调整数组维度顺序：

方法一：对录制得到的data做转置处理

audio_frame = av.AudioFrame.from_ndarray(
    data.T, format='s16', layout='stereo')

方法二：调用sd.rec时指定channels_first=True，直接获取通道在前的数组

data = sd.rec(frames, samplerate=self.rate,
              channels=self.channels, dtype=np.int16, channels_first=True)

两种方法都能让数组形状符合PyAV的要求，解决该报错。

内容的提问来源于stack exchange，提问作者Ismael Hadj