ARKit：基于面部的远距离节点同步移动替代方案咨询

Great question! I’ve run into this exact limitation with ARFaceTrackingConfiguration before—let’s break down why it happens and walk through the viable ARKit alternatives to keep your diamond node synced with facial movement at longer distances.

First, the root cause: ARFaceTrackingConfiguration depends entirely on the TrueDepth camera’s depth sensors, which have a reliable working range of roughly 0.5 to 2 meters. Outside that window, the system can’t capture accurate depth data to detect or track your face, so the renderer delegate stops triggering updates.

Now, here are the best alternatives:

Alternative 1: ARWorldTrackingConfiguration + Vision Framework Face Detection #

This approach combines ARKit’s world tracking (which works at much longer distances) with Apple’s Vision framework to detect your face in the camera feed, then maps that 2D detection to a 3D position in AR space.

How to implement it: #

1. Set up your AR session with ARWorldTrackingConfiguration instead of face tracking:

let configuration = ARWorldTrackingConfiguration()
configuration.planeDetection = .horizontal // Helps with 3D position estimation
session.run(configuration)

2. Use Vision’s VNDetectFaceRectanglesRequest to detect faces in each AR frame:

func session(_ session: ARSession, didUpdate frame: ARFrame) {
    // Process the camera frame with Vision
    let imageHandler = VNImageRequestHandler(cvPixelBuffer: frame.capturedImage, orientation: .right)
    let faceRequest = VNDetectFaceRectanglesRequest { [weak self] request, error in
        guard let self = self, let faceObservations = request.results as? [VNFaceObservation], let face = faceObservations.first else {
            // No face detected—hide or reset your diamond node
            self.diamondNode.isHidden = true
            return
        }
        
        self.diamondNode.isHidden = false
        
        // Convert the 2D face bounding box to a normalized point (center of the face)
        let boundingBox = face.boundingBox
        let normalizedCenter = CGPoint(
            x: boundingBox.midX,
            y: 1 - boundingBox.midY // Vision uses top-left origin; ARKit uses bottom-left
        )
        
        // Use ARKit's raycast to estimate the 3D position of the face
        let raycastQuery = frame.raycastQuery(from: normalizedCenter, allowing: .estimatedPlane, alignment: .horizontal)
        if let raycastResult = session.raycast(raycastQuery).first {
            // Update the diamond node's position to match the estimated face position
            self.diamondNode.transform = raycastResult.worldTransform
        }
    }
    
    // Run the face detection request
    do {
        try imageHandler.perform([faceRequest])
    } catch {
        print("Face detection failed: \(error.localizedDescription)")
    }
}

Pros & Cons: #

• Pros: Works on nearly all ARKit-compatible devices (iOS 11+), supports distances well beyond 2 meters, no dependency on TrueDepth.
• Cons: 3D position accuracy isn’t as precise as ARFaceTracking (especially in environments without horizontal planes), and you may see minor drift over time.

Alternative 2: LiDAR-Enhanced ARWorldTracking (For LiDAR-Equipped Devices) #

If your app targets devices with LiDAR (iPhone 12 Pro/Max, iPhone 13/14/15 Pro/Max, iPad Pro 2020+), you can boost the accuracy of the above approach using LiDAR’s depth data.

How to implement it: #

1. Enable scene reconstruction in your AR configuration to leverage LiDAR:

let configuration = ARWorldTrackingConfiguration()
configuration.sceneReconstruction = .meshWithClassification // Uses LiDAR to build a 3D mesh of the environment
session.run(configuration)

2. Modify the raycast step to use LiDAR’s mesh data for more accurate 3D positioning:

// Replace the raycast query with one that targets the LiDAR mesh
let raycastQuery = frame.raycastQuery(from: normalizedCenter, allowing: .existingPlaneGeometry, alignment: .any)

Pros & Cons: #

• Pros: Far more accurate 3D position estimation than basic world tracking, even in environments without clear planes, minimal drift.
• Cons: Only works on LiDAR-equipped devices, slightly higher computational overhead.

Bonus Tips: #

• To improve performance, limit face detection to every 2-3 frames instead of every frame—this reduces CPU load without noticeable lag.
• If you need to track facial expressions along with position, you can combine Vision’s VNDetectFaceLandmarksRequest with the position estimation to get basic facial feature data.

内容的提问来源于stack exchange，提问作者Mostafa