缩放CMSampleBuffer尺寸避免Broadcast Extension内存超限

崩溃原因排查 #

• 内存管理缺失：CVPixelBuffer属于CoreFoundation类型，ARC不会自动管理其生命周期，你当前代码中从缓冲池申请的pixelBuffer、旧的缓冲池实例没有手动释放，会持续堆积内存；同时输入的原始CMSampleBuffer如果没有及时调用CMSampleBufferInvalidate释放，也会占用大量内存，而Broadcast Upload Extension的内存上限仅为50MB，很容易触发阈值崩溃。
• CIContext配置不合理：默认CIContext会开启中间缓存，且如果没有指定Metal硬件加速，会 fallback 到CPU渲染，产生大量临时内存副本，进一步拉高内存占用。
• 缩放方案开销过高：CILanczosScaleTransform属于高质量缩放滤镜，计算和内存开销都很大，不适合低内存限制的扩展环境；同时你全程使用CIImage做中转，会额外产生多份内存副本。
• 缓冲池配置错误：你创建缓冲池时没有限制最大缓冲数量，系统会按需创建过多pixelBuffer，直接超过内存上限。
• 复用逻辑存在隐患：你使用全局变量持有pixelBuffer复用，若OpenTok的consumeImageBuffer是异步处理逻辑，会出现你正在写入新帧、OpenTok还在读取旧帧的内存冲突，同时旧的pixelBuffer没有及时释放也会造成泄漏。

优化方案 #

1. 补全内存管理逻辑 #

首先实现完整的资源销毁方法，确保旧的缓冲池和pixelBuffer都被正确释放：

private func destroyPixelBuffers() {
    if let pixelBuffer = self.pixelBuffer {
        CVPixelBufferRelease(pixelBuffer)
        self.pixelBuffer = nil
    }
    if let pool = self.pixelBufferPool {
        CVPixelBufferPoolRelease(pool)
        self.pixelBufferPool = nil
    }
}

在processSampleBuffer中处理完CMSampleBuffer后立刻调用CMSampleBufferInvalidate(sampleBuffer)释放原始帧内存，不要长时间持有。

2. 优化CIContext配置 #

关闭中间缓存，优先使用Metal硬件加速：

lazy var context: CIContext = {
    guard let metalDevice = MTLCreateSystemDefaultDevice() else {
        return CIContext(options: [.useSoftwareRenderer: false, .cacheIntermediates: false])
    }
    return CIContext(mtlDevice: metalDevice, options: [.cacheIntermediates: false])
}()

3. 修正缓冲池配置，限制缓冲数量 #

创建缓冲池时添加缓冲数量限制，避免创建过多pixelBuffer：

private func updateBufferPool(newWidth: Int, newHeight: Int) {
    // 先销毁旧缓冲池
    destroyPixelBuffers()
    
    let pixelBufferAttributes: [String: Any] = [
        kCVPixelBufferPixelFormatTypeKey as String: UInt(self.videoFormat),
        kCVPixelBufferWidthKey as String: newWidth,
        kCVPixelBufferHeightKey as String: newHeight,
        kCVPixelBufferIOSurfacePropertiesKey as String: [:]
    ]
    // 限制缓冲池最多保留3个缓冲，满足双/三缓冲需求即可
    let poolAttributes: [String: Any] = [
        kCVPixelBufferPoolMaximumBufferCountKey as String: 3
    ]
    
    CVPixelBufferPoolCreate(nil, poolAttributes as NSDictionary, pixelBufferAttributes as NSDictionary, &pixelBufferPool)
}

4. 重构帧处理逻辑，添加自动释放池 #

每帧处理添加自动释放池，及时销毁临时对象，不要用全局pixelBuffer复用，避免内存冲突：

func processPixelBuffer(pixelBuffer:CVPixelBuffer, timeStamp ts:CMTime) {
    // 每帧加自动释放池，临时对象处理完立刻释放
    autoreleasepool {
        guard  let ciImage = self.scaleFilterImage(inputImage: pixelBuffer.cmIImage, withAspectRatio: 1.0, scale: CGFloat(kVideoFrameScaleFactor)) else {return}
        let targetWidth = Int(ciImage.extent.size.width)
        let targetHeight = Int(ciImage.extent.size.height)
        
        if self.pixelBufferPool == nil {
            self.updateBufferPool(newWidth: targetWidth, newHeight: targetHeight)
        }
        
        var outputPixelBuffer: CVPixelBuffer?
        guard CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, self.pixelBufferPool, &outputPixelBuffer) == kCVReturnSuccess, let outputBuffer = outputPixelBuffer else {return}
        
        // 渲染前后加锁，避免内存访问冲突
        CVPixelBufferLockBaseAddress(outputBuffer, [])
        context?.render(ciImage, to: outputBuffer)
        CVPixelBufferUnlockBaseAddress(outputBuffer, [])
        
        self.videoCaptureConsumer?.consumeImageBuffer(outputBuffer,
                                                      orientation:.up,
                                                      timestamp:ts,
                                                      metadata:nil)
        // 若consumeImageBuffer为异步持有buffer，需等OpenTok回调通知帧处理完成后再释放
        CVPixelBufferRelease(outputBuffer)
    }
}

5. 可选：替换为更轻量的缩放方案 #

如果调整后内存依然过高，可弃用CIFilter缩放，改用Accelerate框架的vImage接口做缩放，内存开销和计算速度都比CIFilter好很多，非常适合广播扩展场景。

内容的提问来源于stack exchange，提问作者Mehmet Baykar