如何将CIFilter输出转换为CMSampleBuffer？求低CPU占用方案

1. 用Metal加速的CIContext替代默认上下文 #

默认CIContext会使用CPU渲染，换成基于Metal的GPU加速上下文能大幅降低CPU占用。建议全局初始化一个单例上下文，避免重复创建：

import CoreImage.Metal
import Metal

// 全局单例，仅初始化一次
let metalDevice = MTLCreateSystemDefaultDevice()
let ciContext = CIContext(mtlDevice: metalDevice!)

2. 复用CVPixelBuffer池减少内存开销 #

每次创建新的CVPixelBuffer会带来额外内存分配开销，复用缓冲区池能避免重复创建，同时匹配原缓冲区的格式、尺寸，减少格式转换损耗：

// 从原CMSampleBuffer提取属性，创建适配的像素缓冲区池
func createPixelBufferPool(from sampleBuffer: CMSampleBuffer) -> CVPixelBufferPool? {
    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return nil }
    
    let attributes = [
        kCVPixelBufferWidthKey: CVPixelBufferGetWidth(pixelBuffer),
        kCVPixelBufferHeightKey: CVPixelBufferGetHeight(pixelBuffer),
        kCVPixelBufferPixelFormatTypeKey: CVPixelBufferGetPixelFormatType(pixelBuffer),
        kCVPixelBufferIOSurfacePropertiesKey: [:] as CFDictionary
    ] as CFDictionary
    
    var pool: CVPixelBufferPool?
    let status = CVPixelBufferPoolCreate(kCFAllocatorDefault, nil, attributes, &pool)
    return status == kCVReturnSuccess ? pool : nil
}

// 从池中获取可用的像素缓冲区
func getPixelBuffer(from pool: CVPixelBufferPool) -> CVPixelBuffer? {
    var pixelBuffer: CVPixelBuffer?
    let status = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, pool, &pixelBuffer)
    return status == kCVReturnSuccess ? pixelBuffer : nil
}

3. 完整低CPU占用处理流程 #

结合上述优化，完整的CMSampleBuffer滤镜处理流程如下：

// 假设已全局初始化ciContext和pixelBufferPool（提前创建一次）
func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with filter: CIFilter) -> CMSampleBuffer? {
    guard let inputPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer),
          let outputPixelBuffer = getPixelBuffer(from: yourPixelBufferPool), // 从复用池取缓冲区
          let inputImage = CIImage(cvImageBuffer: inputPixelBuffer) else {
        return nil
    }
    
    filter.setValue(inputImage, forKey: kCIInputImageKey)
    guard let outputImage = filter.outputImage else { return nil }
    
    // 用Metal上下文GPU加速渲染，CPU占用极低
    ciContext.render(outputImage, to: outputPixelBuffer)
    
    // 基于处理后的像素缓冲区创建新的CMSampleBuffer
    var timingInfo = CMSampleTimingInfo(
        presentationTimeStamp: CMSampleBufferGetPresentationTimeStamp(sampleBuffer),
        duration: CMSampleBufferGetDuration(sampleBuffer),
        decodeTimeStamp: CMSampleBufferGetDecodeTimeStamp(sampleBuffer)
    )
    
    var sampleBufferOut: CMSampleBuffer?
    guard let formatDesc = CMVideoFormatDescriptionCreateForImageBuffer(nil, outputPixelBuffer) else {
        return nil
    }
    let status = CMSampleBufferCreateForImageBuffer(
        kCFAllocatorDefault,
        outputPixelBuffer,
        true,
        nil,
        nil,
        formatDesc,
        &timingInfo,
        &sampleBufferOut
    )
    
    return status == kCVReturnSuccess ? sampleBufferOut : nil
}

// 使用示例
let filter = YUCIHighPassSkinSmoothing()
filter.inputAmount = 0.8
if let processedBuffer = processSampleBuffer(yourOriginalSampleBuffer, with: filter) {
    // 处理后的CMSampleBuffer可直接使用
}

关键说明 #

• output.pixelBuffer返回nil是因为多数CIFilter的输出是延迟计算的虚拟图像，并非直接关联物理像素缓冲区，必须通过CIContext渲染才能得到实际数据。
• 必须复用CIContext和CVPixelBufferPool，禁止在每帧处理时重复创建，这是降低开销的核心。
• 如果原缓冲区是YUV格式，避免转成ARGB处理，直接用CIImage的YUV初始化方法，减少格式转换的性能损耗。

内容的提问来源于stack exchange，提问作者famfamfam