如何在Xcode的RealityKit项目中仅录制ARView相机画面？

Hey there! I’ve been in exactly this situation before—trying to record just the ARView in a RealityKit project without using full-screen ReplayKit or dealing with SceneKit-specific APIs. Here are three solid solutions that work well:

Since RealityKit doesn’t have SceneKit’s renderer callback, we can fall back to capturing regular snapshots of the ARView and stitching them into a video using AVFoundation. Here’s how to pull it off:

• Set up an AVAssetWriter and AVAssetWriterInputPixelBufferAdaptor to handle video file writing
• Use a CADisplayLink (better than Timer for frame sync) to grab snapshots at display refresh rate
• Convert each snapshot to a CVPixelBuffer and feed it into the asset writer
• Don’t forget to handle camera and photo library permissions first!

Here’s a condensed code example to get you started:

import AVFoundation
import RealityKit

class ARViewRecorder {
    private var assetWriter: AVAssetWriter?
    private var pixelBufferAdaptor: AVAssetWriterInputPixelBufferAdaptor?
    private var displayLink: CADisplayLink?
    private let processingQueue = DispatchQueue(label: "com.yourapp.arrecorder")
    private weak var arView: ARView?

    init(arView: ARView) {
        self.arView = arView
    }

    func startRecording(to outputURL: URL) throws {
        guard let arView = arView else { throw RecorderError.missingARView }
        
        assetWriter = try AVAssetWriter(outputURL: outputURL, fileType: .mp4)
        let videoSize = arView.bounds.size
        
        // Configure video input settings
        let videoSettings: [String: Any] = [
            AVVideoCodecKey: AVVideoCodecType.h264,
            AVVideoWidthKey: videoSize.width,
            AVVideoHeightKey: videoSize.height
        ]
        let videoInput = AVAssetWriterInput(mediaType: .video, outputSettings: videoSettings)
        videoInput.expectsMediaDataInRealTime = true
        
        // Set up pixel buffer adaptor
        let pixelBufferAttrs: [String: Any] = [
            kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA,
            kCVPixelBufferWidthKey as String: videoSize.width,
            kCVPixelBufferHeightKey as String: videoSize.height
        ]
        pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(
            assetWriterInput: videoInput,
            sourcePixelBufferAttributes: pixelBufferAttrs
        )

        if assetWriter?.canAdd(videoInput) == true {
            assetWriter?.add(videoInput)
        }

        assetWriter?.startWriting()
        assetWriter?.startSession(atSourceTime: .zero)
        
        // Start capturing frames
        displayLink = CADisplayLink(target: self, selector: #selector(captureCurrentFrame))
        displayLink?.add(to: .main, forMode: .common)
    }

    @objc private func captureCurrentFrame() {
        guard let arView = arView,
              let adaptor = pixelBufferAdaptor,
              let input = adaptor.assetWriterInput,
              input.isReadyForMoreMediaData,
              let snapshot = arView.snapshot() else { return }
        
        processingQueue.async {
            guard let pixelBuffer = snapshot.toPixelBuffer() else { return }
            let timestamp = CMTimeMakeWithSeconds(CACurrentMediaTime(), preferredTimescale: 1000)
            adaptor.append(pixelBuffer, withPresentationTime: timestamp)
        }
    }

    func stopRecording(completion: @escaping () -> Void) {
        displayLink?.invalidate()
        pixelBufferAdaptor?.assetWriterInput.markAsFinished()
        assetWriter?.finishWriting(completionHandler: completion)
    }

    enum RecorderError: Error {
        case missingARView
    }
}

// Helper extension to convert UIImage to CVPixelBuffer
extension UIImage {
    func toPixelBuffer() -> CVPixelBuffer? {
        let attrs = [
            kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue,
            kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue
        ] as CFDictionary
        
        var pixelBuffer: CVPixelBuffer?
        let status = CVPixelBufferCreate(
            kCFAllocatorDefault,
            Int(size.width),
            Int(size.height),
            kCVPixelFormatType_32BGRA,
            attrs,
            &pixelBuffer
        )
        
        guard status == kCVReturnSuccess, let buffer = pixelBuffer else { return nil }
        
        CVPixelBufferLockBaseAddress(buffer, .init(rawValue: 0))
        defer { CVPixelBufferUnlockBaseAddress(buffer, .init(rawValue: 0)) }
        
        guard let context = CGContext(
            data: CVPixelBufferGetBaseAddress(buffer),
            width: Int(size.width),
            height: Int(size.height),
            bitsPerComponent: 8,
            bytesPerRow: CVPixelBufferGetBytesPerRow(buffer),
            space: CGColorSpaceCreateDeviceRGB(),
            bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue
        ), let cgImage = cgImage else { return nil }
        
        context.draw(cgImage, in: CGRect(origin: .zero, size: size))
        return buffer
    }
}

If you don’t want to build a full recording pipeline from scratch, ReplayKit can still work—you just need to crop the full-screen frames down to your ARView’s bounds. Here’s the gist:

• Use RPScreenRecorder.shared().startCapture to get raw CMSampleBuffer frames
• Convert each sample buffer to a CVPixelBuffer
• Crop the pixel buffer to match your ARView’s position and size on screen
• Write the cropped frames to a video file using AVFoundation

This is great if you want to leverage ReplayKit’s optimizations but only keep the ARView portion.

For more low-level control, you can configure RealityKit to render the ARView to a custom Metal texture, then convert those texture frames into video. This requires some Metal knowledge, but it’s the most performant option:

• Create a MTLTexture as a render target
• Assign it to your ARView’s renderTarget property
• Use a MTLCommandQueue to read data from the texture each frame
• Convert the texture data to a video frame and write it out

内容的提问来源于stack exchange，提问作者Tad