60FPS下向CMSampleBuffer插入图片时CIContext.render崩溃求助

问题概述 #

将图片合成到CMSampleBuffer时，代码会随机在ciContext.render(compositedImage, to: outputBuffer)行触发坏内存访问崩溃，且仅在60FPS场景下出现，30FPS时无异常。函数由同一线程的代理直接调用，已尝试Metal/CPU版本CIContext及手动锁定解锁缓冲区操作。

核心问题与修复方案 #

1. 移除不必要的像素缓冲区手动锁定 #

Metal版本的CIContext会自动处理像素缓冲区的访问同步，手动锁定反而可能引发线程冲突，高帧率下更易触发崩溃。

// 删除以下两行代码
// let flag: CVPixelBufferLockFlags = CVPixelBufferLockFlags(rawValue: 0)
// CVPixelBufferLockBaseAddress(outputBuffer, flag)
ciContext.render(compositedImage, to: outputBuffer)
// CVPixelBufferUnlockBaseAddress(outputBuffer, flag)

2. 优化像素缓冲区生命周期管理 #

创建的像素缓冲区可能在CIContext渲染完成前被提前释放，60FPS下频繁的内存回收放大了这个问题。确保CMSampleBuffer持有缓冲区的强引用，或手动保留缓冲区：

guard let outputBuffer = newPixelBuffer else { return nil }
CVPixelBufferRetain(outputBuffer) // 手动保留缓冲区

// 执行渲染和CMSampleBuffer创建操作
ciContext.render(compositedImage, to: outputBuffer)

// 创建CMSampleBuffer后无需手动Release，CMSampleBuffer会管理引用

3. 简化像素缓冲区属性配置 #

冗余的IOSurface属性可能导致内存管理异常，去掉非必要的属性设置：

let pixelBufferAttrs = [
    kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA),
    kCVPixelBufferWidthKey as String: CVPixelBufferGetWidth(pixelBuffer),
    kCVPixelBufferHeightKey as String: CVPixelBufferGetHeight(pixelBuffer)
] as CFDictionary

4. 优化CIContext初始化配置 #

使用Metal设备和专用命令队列初始化CIContext，避免高帧率下的线程调度冲突：

// 全局初始化CIContext（不要在process函数内重复创建）
let metalDevice = MTLCreateSystemDefaultDevice()
let metalQueue = metalDevice?.makeCommandQueue()
let ciContext = CIContext(mtlDevice: metalDevice, commandQueue: metalQueue, options: [
    CIContextOption.useSoftwareRenderer: false,
    CIContextOption.highQualityDownsample: false
])

修改后的完整代码 #

// 全局初始化CIContext
let metalDevice = MTLCreateSystemDefaultDevice()
let metalQueue = metalDevice?.makeCommandQueue()
let ciContext = CIContext(mtlDevice: metalDevice, commandQueue: metalQueue, options: [
    CIContextOption.useSoftwareRenderer: false,
    CIContextOption.highQualityDownsample: false
])

func process(sampleBuffer: CMSampleBuffer) -> CMSampleBuffer? {
    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
        return nil
    }

    let ciImage = CIImage(cvPixelBuffer: pixelBuffer).oriented(.up).clampedToExtent()
    let compositedImage = overlayImage.composited(over: ciImage)

    var newPixelBuffer: CVPixelBuffer?
    let pixelBufferAttrs = [
        kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA),
        kCVPixelBufferWidthKey as String: CVPixelBufferGetWidth(pixelBuffer),
        kCVPixelBufferHeightKey as String: CVPixelBufferGetHeight(pixelBuffer)
    ] as CFDictionary

    CVPixelBufferCreate(kCFAllocatorDefault,
                        CVPixelBufferGetWidth(pixelBuffer),
                        CVPixelBufferGetHeight(pixelBuffer),
                        kCVPixelFormatType_32BGRA,
                        pixelBufferAttrs,
                        &newPixelBuffer)

    guard let outputBuffer = newPixelBuffer else { return nil }
    CVPixelBufferRetain(outputBuffer)

    ciContext.render(compositedImage, to: outputBuffer)
    
    var timingInfo = CMSampleTimingInfo()
    CMSampleBufferGetSampleTimingInfo(sampleBuffer, at: 0, timingInfoOut: &timingInfo)

    var formatDescription: CMFormatDescription?
    CMVideoFormatDescriptionCreateForImageBuffer(allocator: kCFAllocatorDefault,
                                                 imageBuffer: outputBuffer,
                                                 formatDescriptionOut: &formatDescription)

    guard let newFormatDesc = formatDescription else {
        CVPixelBufferRelease(outputBuffer)
        return nil
    }

    var newSampleBuffer: CMSampleBuffer?
    CMSampleBufferCreateReadyWithImageBuffer(allocator: kCFAllocatorDefault,
                                             imageBuffer: outputBuffer,
                                             formatDescription: newFormatDesc,
                                             sampleTiming: &timingInfo,
                                             sampleBufferOut: &newSampleBuffer)

    return newSampleBuffer
}

内容的提问来源于stack exchange，提问作者Peter Lapisu