如何手动为ARSCNView设置iPhone X的前置/后置摄像头作为场景背景?
解决ARSCNView手动设置前置/后置摄像头作为背景的问题
你之前直接给sceneView.scene.background.contents赋值AVCaptureDevice、Input或者PreviewLayer的思路不对——SceneKit的背景内容根本不支持这些类型!它只认颜色、图片、AVPlayer、Metal纹理、CVPixelBuffer这类能直接提供渲染内容的对象。下面分两种常见场景给你具体的实现方案:
场景1:不需要AR功能,仅用ARSCNView展示3D+自定义摄像头背景
这种情况需要自己搭建AVCaptureSession捕获视频帧,转换成Metal纹理后再喂给SceneKit背景,步骤如下:
1. 准备权限
首先在Info.plist里添加摄像头权限描述,否则App会直接崩溃:
<key>NSCameraUsageDescription</key> <string>需要访问摄像头来显示实时背景</string>
2. 完整代码实现
import ARKit import AVFoundation import Metal class YourViewController: UIViewController, ARSCNViewDelegate, AVCaptureVideoDataOutputSampleBufferDelegate { @IBOutlet weak var sceneView: ARSCNView! private var captureSession: AVCaptureSession? private var videoOutput: AVCaptureVideoDataOutput? private var metalDevice: MTLDevice! private var textureCache: CVMetalTextureCache! override func viewDidLoad() { super.viewDidLoad() // 初始化Metal设备和纹理缓存(用于视频帧转纹理) metalDevice = MTLCreateSystemDefaultDevice() CVMetalTextureCacheCreate(nil, nil, metalDevice, nil, &textureCache) // 给ARSCNView加个3D物体示例(你可以替换成自己的场景) let cubeNode = SCNNode(geometry: SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)) cubeNode.position = SCNVector3(0, 0, -0.5) sceneView.scene.rootNode.addChildNode(cubeNode) // 禁用ARSCNView默认的AR会话(我们要自己控制摄像头) sceneView.session.pause() sceneView.scene.background.contents = nil // 启动自定义摄像头会话(默认后置,改成.front就是前置) setupCaptureSession(position: .back) } // 配置AVCaptureSession,切换摄像头就改这里的position参数 private func setupCaptureSession(position: AVCaptureDevice.Position) { let session = AVCaptureSession() session.sessionPreset = .hd1920x1080 // 获取指定位置的摄像头 guard let camera = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: position) else { print("找不到指定位置的摄像头") return } // 添加摄像头输入 do { let input = try AVCaptureDeviceInput(device: camera) if session.canAddInput(input) { session.addInput(input) } } catch { print("摄像头输入初始化失败:\(error.localizedDescription)") return } // 添加视频输出,用于获取实时帧 let output = AVCaptureVideoDataOutput() output.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA] output.alwaysDiscardsLateVideoFrames = true if session.canAddOutput(output) { session.addOutput(output) } captureSession = session videoOutput = output // 设置代理接收视频帧,用后台队列避免卡主线程 videoOutput?.setSampleBufferDelegate(self, queue: DispatchQueue(label: "camera.video.queue")) // 启动会话 session.startRunning() } // 处理捕获到的每一帧视频 func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) { guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return } let width = CVPixelBufferGetWidth(pixelBuffer) let height = CVPixelBufferGetHeight(pixelBuffer) // 把CVPixelBuffer转换成Metal纹理(SceneKit能直接识别) var metalTexture: CVMetalTexture? CVMetalTextureCacheCreateTextureFromImage(nil, textureCache, pixelBuffer, nil, .bgra8Unorm, width, height, 0, &metalTexture) if let metalTex = metalTexture, let texture = CVMetalTextureGetTexture(metalTex) { // 回到主线程设置背景 DispatchQueue.main.async { self.sceneView.scene.background.contents = texture } // 释放纹理资源 CVMetalTextureRelease(metalTex) } } // 切换摄像头的方法(可以绑定到按钮点击事件) @IBAction func switchCameraTapped(_ sender: UIButton) { captureSession?.stopRunning() let currentPosition = (captureSession?.inputs.first as? AVCaptureDeviceInput)?.device.position ?? .back let newPosition = currentPosition == .back ? .front : .back setupCaptureSession(position: newPosition) } }
场景2:需要AR功能,手动切换前置/后置摄像头
如果你的需求是在AR场景中切换摄像头(比如从后置世界追踪切到前置面部追踪),那完全不需要自己搞AVCaptureSession——直接切换ARKit的会话配置就行,ARKit会自动帮你管理摄像头和背景:
// 切换到前置摄像头(面部追踪模式) func switchToFrontCameraForAR() { let faceConfig = ARFaceTrackingConfiguration() faceConfig.isLightEstimationEnabled = true // 重置追踪状态并清除现有锚点 sceneView.session.run(faceConfig, options: [.resetTracking, .removeExistingAnchors]) } // 切换到后置摄像头(世界追踪模式) func switchToBackCameraForAR() { let worldConfig = ARWorldTrackingConfiguration() worldConfig.planeDetection = [.horizontal, .vertical] sceneView.session.run(worldConfig, options: [.resetTracking, .removeExistingAnchors]) }
这种方式下,ARSCNView的背景会自动跟着摄像头切换,同时保留AR的所有功能(比如面部锚点、平面检测)。
内容的提问来源于stack exchange,提问作者mikeymike




