ARKit集成VisionKit人脸检测时人脸框坐标映射错误如何解决

• 坐标系规则不匹配：VNFaceObservation返回的boundingBox为0~1区间的归一化坐标，原点位于处理后图像的左下角，Y轴向上；而CALayer/UIKit坐标系原点位于视图左上角，Y轴向下，直接用视图尺寸乘归一化值必然出现Y轴上下反向偏移。
• 方向参数写死：初始化VNImageRequestHandler时固定传入.leftMirrored方向，未随设备横竖屏状态、前置摄像头镜像规则动态调整，直接导致X轴左右反向、横屏坐标完全失准。
• 内容填充模式适配缺失：ARSCNView默认使用aspectFill内容模式，摄像头原始帧宽高比与视图宽高比不一致时，画面会做居中裁剪，直接用全视图尺寸计算坐标会出现横屏时检测框压缩、位置错位。

1. 新增动态方向获取方法 #

根据当前设备方向，返回Vision识别需要的正确图像方向参数，自动适配前置摄像头镜像规则：

private func currentCaptureOrientation() -> CGImagePropertyOrientation {
    switch UIDevice.current.orientation {
    case .portrait:
        return .rightMirrored
    case .portraitUpsideDown:
        return .leftMirrored
    case .landscapeLeft:
        return .downMirrored
    case .landscapeRight:
        return .upMirrored
    case .faceUp, .faceDown, .unknown:
        return .rightMirrored
    @unknown default:
        return .rightMirrored
    }
}

2. 修正渲染回调逻辑 #

替换原render(_:updateAtTime:)方法，移除写死的方向参数，补全图层清空逻辑避免内存泄漏：

public func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
    // 主线程移除旧的人脸框图层
    DispatchQueue.main.async {
        self.faceLayers.forEach { $0.removeFromSuperlayer() }
        self.faceLayers.removeAll()
    }
    
    guard let capturedImage = sceneView.session.currentFrame?.capturedImage else { return }
    
    let faceDetectionRequest = VNDetectFaceRectanglesRequest { [weak self] request, error in
        guard let self = self, error == nil,
              let observations = request.results as? [VNFaceObservation] else { return }
        DispatchQueue.main.async {
            self.handleFaceDetectionObservations(observations: observations, pixelBuffer: capturedImage)
        }
    }
    
    let imageRequestHandler = VNImageRequestHandler(
        cvPixelBuffer: capturedImage,
        orientation: currentCaptureOrientation(),
        options: [:]
    )
    
    do {
        try imageRequestHandler.perform([faceDetectionRequest])
    } catch {
        print("Vision detection failed: \(error.localizedDescription)")
    }
}

3. 重写坐标映射逻辑 #

替换原handleFaceDetectionObservations方法，适配坐标系翻转、aspectFill裁剪偏移，兼容横竖屏尺寸计算：

fileprivate func handleFaceDetectionObservations(observations: [VNFaceObservation], pixelBuffer: CVPixelBuffer) {
    let viewSize = sceneView.bounds.size
    let bufferWidth = CGFloat(CVPixelBufferGetWidth(pixelBuffer))
    let bufferHeight = CGFloat(CVPixelBufferGetHeight(pixelBuffer))
    let orientation = currentCaptureOrientation()
    
    // 旋转90/270度时，图像显示宽高需要互换
    let isRotatedOrientation = orientation == .left || orientation == .right || orientation == .leftMirrored || orientation == .rightMirrored
    let displayImageWidth = isRotatedOrientation ? bufferHeight : bufferWidth
    let displayImageHeight = isRotatedOrientation ? bufferWidth : bufferHeight
    
    // 计算aspectFill模式下的缩放比例和居中裁剪偏移
    let scaleX = viewSize.width / displayImageWidth
    let scaleY = viewSize.height / displayImageHeight
    let fillScale = max(scaleX, scaleY)
    let scaledImageSize = CGSize(width: displayImageWidth * fillScale, height: displayImageHeight * fillScale)
    let offsetX = (viewSize.width - scaledImageSize.width) / 2
    let offsetY = (viewSize.height - scaledImageSize.height) / 2
    
    for observation in observations {
        let box = observation.boundingBox
        // Y轴翻转：Vision左下原点转UIKit左上原点，X轴已通过正确的orientation参数对齐
        let normalizedRect = CGRect(
            x: box.origin.x,
            y: 1 - box.origin.y - box.height,
            width: box.width,
            height: box.height
        )
        
        // 映射到视图实际显示坐标
        let faceRect = CGRect(
            x: normalizedRect.minX * scaledImageSize.width + offsetX,
            y: normalizedRect.minY * scaledImageSize.height + offsetY,
            width: normalizedRect.width * scaledImageSize.width,
            height: normalizedRect.height * scaledImageSize.height
        )
        
        let faceLayer = CAShapeLayer()
        faceLayer.path = CGPath(rect: faceRect, transform: nil)
        faceLayer.fillColor = UIColor.black.withAlphaComponent(0.3).cgColor
        faceLayer.strokeColor = UIColor.red.cgColor
        faceLayer.lineWidth = 2
        faceLayers.append(faceLayer)
        sceneView.layer.addSublayer(faceLayer)
    }
}

• 检测频率节流：render回调为每帧触发（60次/秒），可添加计数逻辑，每2~3帧执行一次Vision检测，降低CPU占用。
• 原生能力替代：如果仅需要AR场景下的人脸位置，可直接使用ARFaceTrackingConfiguration，框架会自动完成所有坐标映射，无需手动处理坐标系转换。

内容的提问来源于stack exchange，提问作者martin wotterman