如何将RealityView附件视图坐标转换为场景坐标？

完全可以实现SwiftUI附件触摸点到场景空间的坐标映射，核心是解决2D视图坐标到3D实体局部坐标的映射，再转换到场景空间。以下是具体实现方案：

关键步骤 #

• 捕捉视图尺寸与触摸点：在SwiftUI附件视图中记录自身实际尺寸和触摸位置，作为坐标映射的基础
• 归一化触摸坐标：将触摸点从视图像素坐标转换为0-1范围的归一化值，消除视图尺寸差异的影响
• 映射到实体局部空间：根据附件实体的缩放比例，将归一化坐标转换为实体的局部3D坐标
• 转换到场景空间：将实体局部坐标转换为全局场景空间坐标

完整代码示例 #

1. 触摸捕捉SwiftUI视图 #

import SwiftUI
import RealityKit

struct TouchCaptureView: View {
    @Binding var touchPoint: CGPoint?
    @Binding var viewSize: CGSize
    
    var body: some View {
        Color.clear
            .onTapGesture { location in
                touchPoint = location
            }
            .onSizeChange { newSize in
                viewSize = newSize
            }
    }
}

2. RealityView中的坐标转换与交互处理 #

struct ImmersiveView: View {
    @State private var touchPoint: CGPoint?
    @State private var viewSize: CGSize = .zero
    
    var body: some View {
        RealityView { content, attachments in
            // 创建桌面平面实体（示例尺寸：宽1m，深0.6m）
            let deskPlane = ModelEntity(
                mesh: .generatePlane(width: 1, depth: 0.6),
                materials: [SimpleMaterial(color: .gray, isMetallic: false)]
            )
            deskPlane.position = [0, -0.1, 0] // 放置在桌面高度
            content.add(deskPlane)
            
            // 获取并绑定附件实体到桌面平面
            guard let touchAttachment = attachments.entity(for: "touchAttachment") else { return }
            deskPlane.addChild(touchAttachment)
            
            // 调整附件实体的尺寸与位置，匹配桌面平面
            touchAttachment.scale = [1, 1, 0.6] // 和桌面平面尺寸一致
            touchAttachment.position = [0, 0.01, 0] // 轻微抬起避免遮挡桌面
        } update: { content, attachments in
            guard let touchPoint = touchPoint,
                  let attachmentEntity = attachments.entity(for: "touchAttachment"),
                  viewSize != .zero else { return }
            
            // 1. 转换为归一化坐标（SwiftUI Y轴向下，需反转对应RealityKit的Z轴）
            let normalizedX = touchPoint.x / viewSize.width
            let normalizedZ = 1 - (touchPoint.y / viewSize.height)
            
            // 2. 映射到附件实体的局部3D坐标
            // 附件实体scale为[1,1,0.6]，X范围-0.5~0.5，Z范围-0.3~0.3
            let localX = (normalizedX - 0.5) * attachmentEntity.scale.x
            let localZ = (normalizedZ - 0.5) * attachmentEntity.scale.z
            let localPosition = SIMD3<Float>(localX, 0, localZ) // Y=0对应附件实体的平面
            
            // 3. 转换为场景空间坐标
            let scenePosition = attachmentEntity.convert(position: localPosition, to: content)
            
            // 生成并添加球体到触摸对应位置
            let ball = ModelEntity(
                mesh: .generateSphere(radius: 0.02),
                materials: [SimpleMaterial(color: .red, isMetallic: true)]
            )
            ball.position = scenePosition + SIMD3(0, 0.02, 0) // 轻微抬起避免遮挡
            content.add(ball)
            
            // 重置触摸点，防止重复添加
            self.touchPoint = nil
        } attachments: {
            Attachment(id: "touchAttachment") {
                TouchCaptureView(touchPoint: $touchPoint, viewSize: $viewSize)
                    .frame(width: 300, height: 180) // 与实体1:0.6的比例对应
            }
        }
    }
}

你之前的问题分析 #

1. 坐标映射逻辑错误：直接将触摸点的像素值作为3D坐标，未考虑SwiftUI视图尺寸和附件实体的缩放比例，导致坐标偏移
2. 坐标系方向混淆：SwiftUI的Y轴向下，而水平桌面场景用RealityKit的X-Z平面，未做方向反转
3. 转换上下文错误：使用content.convert时未明确附件实体的局部空间，应通过附件实体自身的convert(position:to:)方法完成转换

内容的提问来源于stack exchange，提问作者utahwithak