iOS AudioUnit录制Opus编码UDP传Android爆音问题排查

Hey there, let’s break down what’s likely causing those crackles when sending Opus-encoded audio from iOS to Android, and fix it without modifying your PreferredIOBufferDuration setting. Since raw audio works perfectly, the issue is tied to frame alignment, buffer management, or Opus configuration gaps between capture and playback.

1. Fix Circular Buffer Frame Handling (The Most Likely Culprit) #

Your current logic of waiting for 2 capture frames (341 each = 682 total) then grabbing 640 frames leaves 42 frames unused each cycle. Over time, these leftover frames accumulate and create misaligned, discontinuous audio chunks—this is almost certainly causing the crackles. Instead, you should continuously pull full 640-frame blocks from the circular buffer as soon as enough data is available.

Here’s the adjusted callback code:

func performRecording( _ ioActionFlags: UnsafeMutablePointer<AudioUnitRenderActionFlags>, inTimeStamp: UnsafePointer<AudioTimeStamp>, inBufNumber: UInt32, inNumberFrames: UInt32, ioData: UnsafeMutablePointer<AudioBufferList>) -> OSStatus {
    var err: OSStatus = noErr
    err = AudioUnitRender(audioUnit!, ioActionFlags, inTimeStamp, 1, inNumberFrames, ioData)
    
    if let mData = ioData[0].mBuffers.mData {
        // Write all captured PCM data to the circular buffer
        let bytesWritten = TPCircularBufferProduceBytes(&circularBuffer, mData, inNumberFrames * 2)
        guard bytesWritten == inNumberFrames * 2 else {
            print("Warning: Failed to write all data to circular buffer")
            return err
        }

        var availableBytes: UInt32 = 0
        // Continuously extract full 640-frame blocks until buffer has insufficient data
        while let buffer = TPCircularBufferTail(&circularBuffer, &availableBytes), availableBytes >= 640 * 2 {
            // Copy exactly 640 frames (1280 bytes for Int16) to your target buffer
            memcpy(&targetBuffer, buffer, 640 * 2)
            // Consume the used bytes from the circular buffer
            TPCircularBufferConsume(&circularBuffer, 640 * 2)

            // Encode and send the aligned frame block
            if let encodedData = opusHelper?.encodeStream(of: targetBuffer) {
                let myData = encodedData.withUnsafeBufferPointer { Data(buffer: $0) }
                var protoModel = ProtoModel()
                seqNumber += 1
                protoModel.sequenceNumber = seqNumber
                protoModel.timeStamp = Date().currentTimeInMillis()
                protoModel.payload = myData
                
                DispatchQueue.global().async {
                    do {
                        try self.udpClient?.send(data: protoModel)
                    } catch {
                        print(error.localizedDescription)
                    }
                }
            }
        }
    }
    return err
}

This ensures you never have leftover frames causing audio discontinuities.

2. Validate Opus Encoder/Decoder Configuration #

Since raw audio works, double-check your Opus setup for mismatches:

• Sample Rate & Channel Count: Confirm both iOS encoder and Android decoder use 16kHz, mono (Int16 PCM). Mismatched channels will immediately cause distortion.
• Frame Size Consistency: 640 frames at 16kHz equals 40ms, which is a valid Opus frame size. Make sure your Android decoder expects exactly 40ms frames during initialization.
• Add Critical Opus Settings: You mentioned missing OPUS_SET_BITRATE_REQUEST—add these optimizations for voice:

  // In your OpusHelper initialization
  opus_encoder_ctl(encoder, OPUS_SET_APPLICATION_REQUEST, OPUS_APPLICATION_VOIP)
  opus_encoder_ctl(encoder, OPUS_SET_BITRATE_REQUEST, 24000) // 24kbps is ideal for voice
  opus_encoder_ctl(encoder, OPUS_SET_INBAND_FEC_REQUEST, 1) // Enable forward error correction for UDP
  

  Forward error correction helps recover from minor UDP packet loss, which can also cause crackles.

3. Add a Jitter Buffer on Android #

UDP is unreliable—even with perfect encoding, network jitter or packet loss can lead to crackles. Raw audio is more forgiving, but Opus-encoded data needs consistent playback timing. Implement a small jitter buffer (e.g., 80-120ms of cached frames) on the Android side:

• Buffer incoming frames by their timestamp/sequence number
• Only start playback once the buffer has enough frames to smooth out delays
• Drop or interpolate missing frames (instead of skipping) to avoid abrupt gaps

4. Verify AudioUnit Capture Format #

Ensure your AudioUnit is capturing in the exact format Opus expects. Confirm your input stream description matches this:

var inputFormat = AudioStreamBasicDescription()
inputFormat.mSampleRate = 16000.0
inputFormat.mFormatID = kAudioFormatLinearPCM
inputFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked
inputFormat.mChannelsPerFrame = 1
inputFormat.mBitsPerChannel = 16
inputFormat.mBytesPerPacket = 2
inputFormat.mBytesPerFrame = 2
inputFormat.mFramesPerPacket = 1

A mismatch here would cause Opus to encode corrupted audio, even if raw playback works.

内容的提问来源于stack exchange，提问作者shayan