VIPER架构下Core Image微笑检测图像缓冲区停止生成问题

场景复现 #

在VIPER架构中基于Core Image实现微笑检测，各模块代码如下：

ViewController 代码 #

负责捕获TrueDepth相机的视频帧与深度数据，传递给Presenter：

extension ViewController: AVCaptureDataOutputSynchronizerDelegate {
    func dataOutputSynchronizer(_ synchronizer: AVCaptureDataOutputSynchronizer, didOutput synchronizedDataCollection: AVCaptureSynchronizedDataCollection) {
        guard let syncedDepthData: AVCaptureSynchronizedDepthData =
                synchronizedDataCollection.synchronizedData(for: multifaceCamera.depthDataOutput) as? AVCaptureSynchronizedDepthData,
              let videoDataOutput = self.videoDataOutput,
              let syncedVideoData: AVCaptureSynchronizedSampleBufferData =
                synchronizedDataCollection.synchronizedData(for: videoDataOutput) as? AVCaptureSynchronizedSampleBufferData else {
            return
        }
        
        if syncedDepthData.depthDataWasDropped || syncedVideoData.sampleBufferWasDropped {
            return
        }
        
        let depthData = syncedDepthData.depthData
        let _depthData = depthData.converting(toDepthDataType: kCVPixelFormatType_DepthFloat16)
        let depthPixelBuffer = _depthData.depthDataMap
        let sampleBuffer = syncedVideoData.sampleBuffer
        guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
            return
        }

        Task {
            await presenter.sendOutput(output: imageBuffer, depthPixelBuffer: depthPixelBuffer)
        }
    }
}

Presenter 代码 #

仅负责将数据转发给Interactor：

func sendOutput(output: CVPixelBuffer, depthPixelBuffer: CVPixelBuffer?) async {
    await interactor.sendOutput(output: output, depthPixelBuffer: depthPixelBuffer)
}

Interactor 代码 #

实现Core Image微笑检测逻辑：

func sendOutput(output: CVPixelBuffer, depthPixelBuffer: CVPixelBuffer?) async {
    let image = CIImage(cvPixelBuffer: output)
    let result = await isSmiling(image: image)
    /// 处理检测结果
}

func isSmiling(image: CIImage) async -> Bool {
    let detector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: [CIDetectorAccuracy: CIDetectorAccuracyHigh, CIDetectorSmile: true])
    let faces = detector?.features(in: image, options: [CIDetectorSmile : true as AnyObject]) as? [CIFaceFeature]
    if let facesFound = faces {
        for face in facesFound {
            return face.hasSmile
        }
    } else {
        return false
    }
    
    return false
}

问题描述 #

• Interactor中的detector?.features(in: image, options: ...)执行几次后完全停止，ViewController不再生成图像缓冲区
• 无崩溃或报错日志，仅控制台输出提示：The input tensor width must be aligned to 64-byte
• 对比测试：使用Vision框架的VNDetectFaceLandmarksRequest可正常检测；将isSmiling方法直接移至ViewController中也能正常运行
• 已尝试无效方案：设置AVCaptureVideoDataOutput像素格式为kCVPixelFormatType_64RGBALE、选择宽度可被16/64整除的设备格式作为active format

问题原因与解决方案 #

核心原因分析 #

1. CIDetector重复创建：每次调用isSmiling都新建CIDetector实例，会导致资源占用异常，甚至触发内部状态错误
2. 像素缓冲区生命周期问题：CVPixelBuffer在传递过程中可能被提前释放，导致Core Image处理中断
3. 线程调度冲突：Task异步传递数据可能导致Core Image与AVCapture的线程上下文冲突，中断帧捕获流程

针对性修复方案 #

1. 复用CIDetector实例

将CIDetector作为Interactor的属性初始化一次，避免重复创建：

class YourInteractor {
    // 全局复用的人脸检测器
    private let faceDetector: CIDetector
    
    init() {
        let detectorOptions: [String: Any] = [
            CIDetectorAccuracy: CIDetectorAccuracyHigh,
            CIDetectorSmile: true
        ]
        faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: detectorOptions)!
    }
    
    // ...其他代码
}

2. 管理像素缓冲区生命周期

在处理CVPixelBuffer时手动延长其生命周期，避免提前释放：

func sendOutput(output: CVPixelBuffer, depthPixelBuffer: CVPixelBuffer?) async {
    // 保留像素缓冲区，确保处理过程中不被释放
    CVPixelBufferRetain(output)
    defer {
        // 处理完成后释放
        CVPixelBufferRelease(output)
    }
    
    let image = CIImage(cvPixelBuffer: output)
    let result = await isSmiling(image: image)
    /// 处理检测结果
}

3. 强制图像尺寸对齐

手动调整CIImage的宽度为64的整数倍，解决控制台提示的对齐问题：

private func alignImageTo64ByteBoundary(image: CIImage) -> CIImage {
    let originalWidth = image.extent.width
    // 计算最近的64倍数宽度
    let alignedWidth = ceil(originalWidth / 64) * 64
    let scaleFactor = alignedWidth / originalWidth
    
    // 缩放图像至对齐尺寸
    return image.transformed(by: CGAffineTransform(scaleX: scaleFactor, y: scaleFactor))
}

// 在sendOutput中调用对齐方法
func sendOutput(output: CVPixelBuffer, depthPixelBuffer: CVPixelBuffer?) async {
    CVPixelBufferRetain(output)
    defer { CVPixelBufferRelease(output) }
    
    let image = CIImage(cvPixelBuffer: output)
    let alignedImage = alignImageTo64ByteBoundary(image: image)
    let result = await isSmiling(image: alignedImage)
    /// 处理检测结果
}

4. 简化检测逻辑

优化isSmiling方法的代码逻辑，提升效率：

func isSmiling(image: CIImage) async -> Bool {
    let faces = faceDetector.features(in: image, options: [CIDetectorSmile: true]) as? [CIFaceFeature]
    // 直接返回第一张人脸的微笑状态，无脸则返回false
    return faces?.first?.hasSmile ?? false
}

验证效果 #

应用上述修改后，Core Image的微笑检测可持续稳定运行，不再出现帧捕获中断的情况，控制台的对齐提示也会消失。

内容的提问来源于stack exchange，提问作者Kevvv