如何在Xcode项目中实现AR虚拟按钮及非触屏交互功能？

Absolutely! This is a super common (and awesome) use case for AR experiences—hands-free interaction with on-screen True/False buttons is totally doable in Xcode, and there are a few robust, user-friendly approaches to implement it. Let’s dive into the most practical options tailored to your project:

1. ARKit Hand Tracking (Best for "Touch-like" Interaction) #

If your target devices run iOS 14 or later, ARKit’s built-in hand tracking is your best bet. It lets you detect real-world hand movements directly in the camera feed, so users can "tap" virtual buttons by touching them with their finger in 3D space. Here’s how to pull it off:

• Enable Hand Tracking: Start with an ARWorldTrackingConfiguration and enable hand tracking frame semantics:

  let configuration = ARWorldTrackingConfiguration()
  configuration.frameSemantics.insert(.handTracking)
  sceneView.session.run(configuration)
  

• Track Finger Position: Use the ARSessionDelegate to listen for hand anchors, then extract the tip position of the index finger (or whichever finger you want users to use):

  func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
      for anchor in anchors {
          guard let handAnchor = anchor as? ARHandAnchor else { continue }
          // Get index finger tip position from the hand skeleton
          let indexTipJoint = handAnchor.skeleton.joints[.fingerTip(.index)]
          let tipWorldPosition = indexTipJoint.localTransform.columns.3
          let tipScenePosition = sceneView.scene.rootNode.convertPosition(
              SCNVector3(tipWorldPosition.x, tipWorldPosition.y, tipWorldPosition.z),
              from: handAnchor
          )
          // Check if the tip intersects with your virtual button
          checkForButtonTap(at: tipScenePosition)
      }
  }
  

• Collision Detection: Implement checkForButtonTap(at:) to detect if the finger tip is within the button’s 3D bounds. For better accuracy, you can use SceneKit’s physics engine or simple bounding box checks, plus add visual feedback to confirm the tap:

  func checkForButtonTap(at position: SCNVector3) {
      // Assume your buttons are named "trueButton" and "falseButton"
      if let trueButton = sceneView.scene.rootNode.childNode(withName: "trueButton", recursively: true),
         trueButton.boundingBox.contains(position) {
          triggerTrueButtonAction()
          // Add visual feedback (e.g., scale animation)
          SCNTransaction.begin()
          SCNTransaction.duration = 0.1
          trueButton.scale = SCNVector3(0.9, 0.9, 0.9)
          SCNTransaction.completionBlock = {
              SCNTransaction.begin()
              SCNTransaction.duration = 0.1
              trueButton.scale = SCNVector3(1.0, 1.0, 1.0)
              SCNTransaction.commit()
          }
          SCNTransaction.commit()
      }
      // Repeat the same logic for the false button
  }
  

2. Face Tracking + Gaze Interaction (Great for Hands-Busy Scenarios) #

If your users have devices with a TrueDepth camera (iPhone X and later), you can use ARKit’s face tracking to let users trigger buttons by gazing at them. This works well if users can’t use their hands:

• Enable Face Tracking: Use ARFaceTrackingConfiguration instead of world tracking:

  let configuration = ARFaceTrackingConfiguration()
  sceneView.session.run(configuration)
  

• Track Gaze Direction: In the ARSessionDelegate, get the ARFaceAnchor’s lookAtPoint—this vector represents where the user is looking in 3D space. Calculate if the gaze is directed at your button (using vector dot product to check alignment), and trigger the action after a short delay (to avoid accidental taps):

  var gazeTimer: Timer?
  func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
      guard let faceAnchor = anchors.first as? ARFaceAnchor else { return }
      let gazeDirection = faceAnchor.lookAtPoint
      // Convert gaze direction to a ray from the camera
      let cameraPosition = sceneView.pointOfView!.position
      let gazeRayEnd = cameraPosition + gazeDirection * 10 // Extend the ray 10m forward
      
      // Check if the ray intersects your button node
      if let trueButton = sceneView.scene.rootNode.childNode(withName: "trueButton", recursively: true),
         trueButton.intersectsRay(from: cameraPosition, to: gazeRayEnd) {
          gazeTimer?.invalidate()
          gazeTimer = Timer.scheduledTimer(withTimeInterval: 0.5, repeats: false) { _ in
              self.triggerTrueButtonAction()
          }
      } else {
          gazeTimer?.invalidate()
      }
  }
  

3. Vision Framework Hand Pose Recognition (Flexible for Custom Gestures) #

If you want more control over gesture types (e.g., a "pinch" instead of a tap), Apple’s Vision framework can detect hand poses in the camera feed. You can combine this with ARKit to map the 2D gesture position to 3D space in your AR scene. The core steps are:

• Request hand pose detection with VNDetectHumanHandPoseRequest
• Convert the detected 2D finger tip position to a 3D point in your AR scene using sceneView.unprojectPoint(_:)
• Check if this 3D point intersects with your virtual buttons

Key Tips for Success #

• Device Compatibility: Always check if the device supports your chosen feature before enabling it. For example:

  if ARWorldTrackingConfiguration.supportsFrameSemantics(.handTracking) {
      // Enable hand tracking
  } else {
      // Fall back to another method or show an error
  }
  

• Clear Visual Feedback: Users need to know when their action has registered—add animations, color changes, or particle effects to your buttons when they’re triggered.
• Calibration: Add a short onboarding step to let users adjust to the interaction (e.g., "Hold your finger in front of the camera to tap the buttons").

内容的提问来源于stack exchange，提问作者abhi