NAudio转Opus编码推流Discord：音频质量与速度异常问题求助

通过URL获取音频，使用NAudio处理后转换为Opus格式推流至Discord时，出现音频质量严重下降、音量丢失、播放速度快数倍的问题，需修复上述问题并提升传输音频质量。

原代码 #

public async Task StreamAudioAsync(string url, SocketMessage msg)
{
    try
    {
        // Create audioClient
        var audioClient = await (msg.Author as IGuildUser)?.VoiceChannel.ConnectAsync();
        
        // Create a resampler to convert MP3 to PCM
        using (var mp3Reader = new MediaFoundationReader(url))
        {
            var resampler = new WdlResamplingSampleProvider(mp3Reader.ToSampleProvider(), 48000);
            // Create an instance of the OpusEncoder class with the desired parameters
            var opusEncoder = new OpusEncoder(48000, 2, Concentus.Enums.OpusApplication.OPUS_APPLICATION_AUDIO);
            opusEncoder.Bitrate = 96000; // Set the bitrate to 96 kbps.

            // Create a stream that accepts data encoded in Opus
            using (var opusStream = audioClient.CreateOpusStream())
            {
                try
                {
                    // Create a buffer to store audio data
                    float[] buffer = new float[3840];
                    int byteCount;

                    // Read audio data from the resampler
                    while ((byteCount = resampler.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        // Create a byte array that is four times the size of the float array
                        var byteBuffer = new byte[byteCount * 4];

                        // Convert each element of the float array into a byte array and copy it into byteBuffer
                        for (int i = 0; i < byteCount; i++)
                        {
                            byte[] bytes = BitConverter.GetBytes(buffer[i]);
                            Array.Copy(bytes, 0, byteBuffer, i * 4, 4);
                        }

                        // Create an instance of the WaveBuffer class using the byte array
                        var waveBuffer = new WaveBuffer(byteBuffer);

                        // Access the float array, which represents the audio signal
                        var floatBuffer = waveBuffer.FloatBuffer;

                        // Determine the frame size depending on the sampling rate
                        // int frameSize = 960 * resampler.WaveFormat.SampleRate / 48000;
                        int frameSize = 960; // it can't be higher than 960 or lower (I don't know what it is).
                        // Console.WriteLine($"framesize :{frameSize}");

                        // Encode audio data into Opus format
                        var opusPacket = new byte[2000]; // Maximum Opus packet size
                        int len = opusEncoder.Encode(floatBuffer, 0, frameSize, opusPacket, 0, opusPacket.Length);


                        // Send audio data to the Opus stream
                        await opusStream.WriteAsync(opusPacket, 0, len);
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine(e);
                }
            }
        }
    }
    catch (Exception e)
    {
        Console.WriteLine("test exception: "+e);
        Console.WriteLine();
    }
}

问题原因分析 #

• 播放速度异常：硬编码固定frameSize=960，未考虑声道数与实际读取的数据量。Opus编码器的frameSize参数是单声道的采样点数量，双声道时输入的音频数据长度应为frameSize × 声道数，原代码每次只编码960个采样点（不足一帧双声道数据），导致播放速度翻倍。
• 音量丢失与质量下降：冗余的float与byte数组互转操作会导致音频精度损失；未根据音频类型优化Opus编码器的应用场景，且未处理读取到的剩余音频数据，导致音频截断。
• 声道不匹配：重采样器未确保输出声道数与Opus编码器一致，若原音频为单声道，直接按双声道编码会导致数据错位。

修正后的代码 #

public async Task StreamAudioAsync(string url, SocketMessage msg)
{
    try
    {
        var guildUser = msg.Author as IGuildUser;
        if (guildUser?.VoiceChannel == null)
        {
            Console.WriteLine("用户未在语音频道中");
            return;
        }

        // 连接到语音频道
        var audioClient = await guildUser.VoiceChannel.ConnectAsync();
        
        using (var mp3Reader = new MediaFoundationReader(url))
        {
            var sourceProvider = mp3Reader.ToSampleProvider();
            // 确保输出为48kHz采样率、双声道（Discord要求的音频格式）
            var resampler = new WdlResamplingSampleProvider(sourceProvider, 48000);
            if (resampler.WaveFormat.Channels != 2)
            {
                resampler = new WdlResamplingSampleProvider(sourceProvider.ToMono().ToStereo(), 48000);
            }

            // 初始化Opus编码器：48kHz、双声道、音乐场景（音质更优）
            var opusEncoder = new OpusEncoder(48000, 2, Concentus.Enums.OpusApplication.OPUS_APPLICATION_MUSIC);
            opusEncoder.Bitrate = 128000; // 提升比特率至128kbps，进一步优化音质
            opusEncoder.Vbr = true; // 启用可变比特率
            
            using (var opusStream = audioClient.CreateOpusStream())
            {
                // Opus 20ms帧长对应的单声道采样数：48000 * 0.02 = 960，双声道则为960*2=1920
                int frameSizePerChannel = 960;
                int totalFrameSize = frameSizePerChannel * 2;
                float[] buffer = new float[totalFrameSize * 2]; // 双帧缓冲区，提升读取效率
                int bytesRead;
                int remainingSamples = 0;

                while ((bytesRead = resampler.Read(buffer, remainingSamples, buffer.Length - remainingSamples)) > 0)
                {
                    remainingSamples += bytesRead;
                    
                    // 按完整帧批量编码
                    while (remainingSamples >= totalFrameSize)
                    {
                        byte[] opusPacket = new byte[4096]; // 增大缓冲区避免溢出
                        int encodedLength = opusEncoder.Encode(
                            buffer, 
                            0, 
                            frameSizePerChannel, // Concentus要求传入单声道的采样数
                            opusPacket, 
                            0, 
                            opusPacket.Length
                        );

                        if (encodedLength > 0)
                        {
                            await opusStream.WriteAsync(opusPacket, 0, encodedLength);
                        }

                        // 将剩余未编码的数据移到缓冲区开头
                        Array.Copy(buffer, totalFrameSize, buffer, 0, remainingSamples - totalFrameSize);
                        remainingSamples -= totalFrameSize;
                    }
                }

                // 处理最后剩余的不足一帧的数据（补零填充）
                if (remainingSamples > 0)
                {
                    Array.Fill(buffer, 0f, remainingSamples, totalFrameSize - remainingSamples);
                    byte[] opusPacket = new byte[4096];
                    int encodedLength = opusEncoder.Encode(
                        buffer, 
                        0, 
                        frameSizePerChannel, 
                        opusPacket, 
                        0, 
                        opusPacket.Length
                    );

                    if (encodedLength > 0)
                    {
                        await opusStream.WriteAsync(opusPacket, 0, encodedLength);
                    }
                }
            }
        }
    }
    catch (Exception e)
    {
        Console.WriteLine($"推流异常: {e.Message}");
    }
}

关键修正说明 #

• 移除冗余转换：删除了无意义的float与byte数组互转操作，直接使用重采样后的float缓冲区，避免音频精度损失。
• 匹配音频格式：确保输出音频为Discord要求的48kHz、双声道格式，单声道音频自动转为双声道。
• 正确帧大小计算：采用Opus标准20ms帧长，计算出对应单声道采样数为960，双声道数据长度为1920，确保编码数据量与帧大小匹配，修复播放速度异常。
• 优化Opus配置：将应用场景设为OPUS_APPLICATION_MUSIC以优化音乐音质，提升比特率至128kbps并启用可变比特率，进一步提升音频质量。
• 处理剩余数据：编码完成所有完整帧后，对剩余不足一帧的数据补零填充并编码，避免音频截断。

内容的提问来源于stack exchange，提问作者TABURELTER