如何将CVPixelBuffer转换为RTCVideoFrameBuffer以创建RTCVideoFrame？

Hey there! I’ve worked extensively with WebRTC’s iOS SDK, so I can help you wrap your CVPixelBuffer into an RTCVideoFrameBuffer smoothly. WebRTC provides a ready-made class for this exact use case—RTCCVPixelBuffer—which is a concrete implementation of RTCVideoFrameBuffer. Here’s how to do it in both Swift and Objective-C:

Swift Implementation #

First, make sure you’ve imported the WebRTC framework:

import WebRTC

Then, convert your CVPixelBuffer and initialize the RTCVideoFrame:

// Assume you already have your input CVPixelBuffer instance
guard let inputPixelBuffer = yourCVPixelBufferSource else {
    // Handle nil case (e.g., log an error, exit early)
    print("Error: CVPixelBuffer is nil")
    return
}

// Wrap the CVPixelBuffer into RTCCVPixelBuffer (which conforms to RTCVideoFrameBuffer)
let videoFrameBuffer = RTCCVPixelBuffer(pixelBuffer: inputPixelBuffer)

// Initialize your RTCVideoFrame with the buffer, rotation, and timestamp
let videoFrame = RTCVideoFrame(
    buffer: videoFrameBuffer,
    rotation: ._0, // Adjust based on your video's orientation: ._90, ._180, ._270
    timeStampNs: Int64(Date().timeIntervalSince1970 * 1_000_000_000)
)

Objective-C Implementation #

Import the WebRTC header first:

#import <WebRTC/WebRTC.h>

Then perform the conversion and frame initialization:

// Assume you have your input CVPixelBufferRef
CVPixelBufferRef inputPixelBuffer = yourCVPixelBufferSource;
if (!inputPixelBuffer) {
    // Handle nil case
    NSLog(@"Error: CVPixelBuffer is nil");
    return;
}

// Wrap the CVPixelBuffer into RTCCVPixelBuffer
RTCCVPixelBuffer *videoFrameBuffer = [[RTCCVPixelBuffer alloc] initWithPixelBuffer:inputPixelBuffer];

// Create the RTCVideoFrame
RTCVideoFrame *videoFrame = [[RTCVideoFrame alloc] initWithBuffer:videoFrameBuffer
                                                          rotation:RTCVideoRotation_0
                                                      timeStampNs:(int64_t)([[NSDate date] timeIntervalSince1970] * 1000000000)];

Key Notes to Keep in Mind #

• Supported Pixel Formats: WebRTC works best with common formats like kCVPixelFormatType_420YpCbCr8BiPlanarFullRange (YUV420) or kCVPixelFormatType_32BGRA. If your input buffer uses an unsupported format, you’ll need to convert it first—you can use Core Image or vImage for format conversion.
• Timestamp Accuracy: The timeStampNs parameter requires a nanosecond-level timestamp. Using Date().timeIntervalSince1970 * 1e9 is a standard way to get this, but make sure it’s synchronized with your video stream’s timing to avoid playback desync.
• Frame Resizing: If you need to adjust the frame size, you can use the resize(to:) method on RTCVideoFrameBuffer to get a scaled version:

  let resizedBuffer = videoFrameBuffer.resize(to: CGSize(width: 640, height: 480))
  

• Memory Management: In ARC-enabled projects (most modern iOS apps), RTCCVPixelBuffer will handle memory automatically. For manual memory management in Objective-C, remember to release the buffer when you’re done with it.

内容的提问来源于stack exchange，提问作者AppleDeveloper