如何在RealityKit（非AR模式）中结合SwiftUI用手势控制相机移动？

Absolutely! You can totally implement gesture-driven 3D camera movement in a non-AR RealityKit + SwiftUI setup — it just takes explicitly setting up the right camera controller (since the default AR-focused controls don’t activate for static non-AR scenes). Let’s walk through the most straightforward approaches:

Using SwiftUI's RealityView (iOS 16+) #

If you’re targeting iOS 16 or later, SwiftUI’s native RealityView is the cleanest way to go. You’ll want to use the OrbitCameraController, which is built specifically for non-AR/static 3D scenes and supports all the standard navigation gestures (drag to rotate, pinch to zoom, pan to reposition).

Here’s a complete example:

import SwiftUI
import RealityKit

struct NonAR3DView: View {
    var body: some View {
        RealityView { content, attachments in
            // Add your 3D content (example: a blue cube)
            let cubeMesh = MeshResource.generateBox(size: 0.5)
            let cubeMaterial = SimpleMaterial(color: .blue, isMetallic: true)
            let cubeEntity = ModelEntity(mesh: cubeMesh, materials: [cubeMaterial])
            content.add(cubeEntity)
            
            // Set up the orbit camera controller for gesture control
            let orbitController = OrbitCameraController()
            orbitController.enableRotation = true  // Drag to rotate view
            orbitController.enableZoom = true      // Pinch to zoom in/out
            orbitController.enablePan = true       // Drag to reposition the camera's target
            orbitController.cameraDistance = 2.0   // Initial distance from the cube
            orbitController.targetPosition = cubeEntity.position  // Point camera at the cube
            
            // Assign the controller and enable non-AR mode
            content.cameraControllers.default = orbitController
            content.cameraMode = .nonAR
        }
        .edgesIgnoringSafeArea(.all)
    }
}

The OrbitCameraController handles all the gesture logic out of the box — no extra work needed to map gestures to camera movement.

Using ARView Wrapped in SwiftUI (iOS 15+) #

If you need to support older iOS versions (15 or earlier), or prefer working with ARView, you can wrap it in a UIViewRepresentable and configure it for non-AR use:

import SwiftUI
import RealityKit
import ARKit

struct ARViewNonARWrapper: UIViewRepresentable {
    func makeUIView(context: Context) -> ARView {
        let arView = ARView(frame: .zero)
        
        // Disable automatic AR session config to force non-AR mode
        arView.automaticallyConfigureSession = false
        let nonARConfig = ARWorldTrackingConfiguration()
        nonARConfig.planeDetection = []
        nonARConfig.environmentTexturing = .none
        arView.session.run(nonARConfig, options: [.resetTracking, .removeExistingAnchors])
        
        // Add your 3D content (example: a red cube)
        let cubeMesh = MeshResource.generateBox(size: 0.5)
        let cubeMaterial = SimpleMaterial(color: .red, isMetallic: false)
        let cubeEntity = ModelEntity(mesh: cubeMesh, materials: [cubeMaterial])
        arView.scene.addAnchor(cubeEntity)
        
        // Set up orbit camera controller for gestures
        let orbitController = OrbitCameraController()
        orbitController.enableRotation = true
        orbitController.enableZoom = true
        orbitController.enablePan = true
        orbitController.cameraDistance = 2.0
        arView.cameraControllers.default = orbitController
        
        return arView
    }
    
    func updateUIView(_ uiView: ARView, context: Context) {}
}

// Use the wrapper in your SwiftUI view
struct ContentView: View {
    var body: some View {
        ARViewNonARWrapper()
            .edgesIgnoringSafeArea(.all)
    }
}

This setup mimics the RealityView approach but uses the older ARView component.

Customizing Gestures (If You Need More Control) #

If the default orbit controller’s gestures don’t fit your needs, you can add custom SwiftUI gestures and manually update the camera’s transform or controller settings. For example, here’s how to add a custom drag gesture to pan the camera’s target:

@State private var lastDragPosition: CGPoint?

RealityView { content, attachments in
    // ... existing setup code for cube and orbit controller
}
.gesture(DragGesture()
    .onChanged { value in
        guard let orbitController = content.cameraControllers.default as? OrbitCameraController else { return }
        if let lastPos = lastDragPosition {
            // Convert 2D drag delta to 3D translation (adjust the multiplier to control sensitivity)
            let deltaX = Float(value.location.x - lastPos.x) * -0.01
            let deltaY = Float(value.location.y - lastPos.y) * 0.01
            orbitController.targetPosition += SIMD3(deltaX, deltaY, 0)
        }
        lastDragPosition = value.location
    }
    .onEnded { _ in
        lastDragPosition = nil
    }
)

Key Takeaway #

Non-AR RealityKit + SwiftUI fully supports gesture-based view navigation — the key is to explicitly use the OrbitCameraController (or a custom camera controller) and enable the gesture flags you need. Both RealityView and wrapped ARView setups work seamlessly for this.

内容的提问来源于stack exchange，提问作者joerup2004