Three.js+WebXR+Vue3 AR场景截图异常：移动端空白/桌面仅显模型

我正在用Three.js、WebXR和Vue.js 3开发增强现实（AR）应用，想添加一个按钮点击捕获当前场景快照，但遇到以下问题：

• 移动端捕获的图像为空白
• 桌面端通过Firefox/Chrome搭配WebXR API扩展模拟Galaxy S8环境时，截图仅显示3D模型，背景为白色

Vue组件模板代码 #

<template>
    <canvas ref="canvas" id="canvas"></canvas>
    <div ref="overlay" id="overlay">
        <button ref="picture" class="picture-button" v-show="arSessionStarted">
        TA BILD
        </button>
        <button ref="rotate" class="rotation-button" v-show="arSessionStarted">
        ROTERA 
        </button>
    </div>
</template>

初始化函数代码 #

init() {
    // 创建THREE场景
    this.scene = new THREE.Scene()
    this.camera = new THREE.PerspectiveCamera(
        70,
        window.innerWidth / window.innerHeight,
        0.1,
        20
    );
    this.camera.position.z = 1;
   // 创建THREE WebGLRenderer
    this.canvas = this.$refs.canvas;
    this.gl = this.canvas.getContext('webgl', {xrCompatible: true});
    this.renderer = new THREE.WebGLRenderer({
        canvas: this.canvas,
        context: this.gl,
        antialias: true,
        alpha: true,
        powerPreference: 'high-performance', 
        xrCompatible: true,
        preserveDrawingBuffer: true
    });
    this.renderer.setPixelRatio(window.devicePixelRatio);
    this.renderer.setSize(window.innerWidth, window.innerHeight);
    this.renderer.xr.enabled = true;

    // 创建THREE灯光对象
    const light = new THREE.HemisphereLight(0xffffff, 0xbbbbff, 1);
    light.position.set(0.5, 1, 0.25);
    this.scene.add(light);
    
    // 加载模型和瞄准器到场景
    this.addReticleToScene();
    this.addModelToScene(); 
    // 创建THREE控制器放置模型
    this.controller = this.renderer.xr.getController(0);
    this.controller.addEventListener("select", this.onSelect);
    this.scene.add(this.controller);
    
    // 创建进入AR会话的按钮
    this.overlay = this.$refs.overlay;
    const button = ARButton.createButton(this.renderer, {
        requiredFeatures: ["hit-test"],
        optionalFeatures: [
            "dom-overlay",
            "dom-overlay-for-handheld-ar",
        ],
        domOverlay: {
            root: this.overlay,
        },
    });
    this.overlay.appendChild(button);
    // 给AR按钮添加ID
    button.id = "my-id";
    // 为旋转和拍照按钮初始化事件监听器
    this.rotationButton = this.$refs.rotate;
    this.takePictureButton = this.$refs.picture;
    this.rotationButton.addEventListener('beforexrselect', ev => ev.preventDefault());
    this.takePictureButton.addEventListener('beforexrselect', ev => ev.preventDefault());
    
    this.rotationButton.addEventListener('click', () => {
        this.rotate();
    });
    
    this.takePictureButton.addEventListener('click', () => {
        this.takePicture();
    });
    // 添加窗口大小调整的事件监听器
    window.addEventListener("resize", this.onWindowResize, false);
}

拍照函数代码 #

takePicture() {
    // 拍照
    this.screenshoot = this.renderer.domElement.toDataURL();
    console.log(this.screenshoot);
    alert("Work in Progress!");
},

渲染相关代码 #

render(timestamp, frame) {
    if (frame) {
        if (!this.hitTestSourceInitialized) {
            this.initializeHitTestSource();
        }
        if (this.hitTestSourceInitialized) {
            const hitTestResults = frame.getHitTestResults(
                this.hitTestSource
            );
            if (hitTestResults.length > 0) {
                const hit = hitTestResults[0];
                const pose = hit.getPose(this.localSpace);
                this.reticle.visible = true;
                this.reticle.matrix.fromArray(pose.transform.matrix);
            } else {
                this.reticle.visible = false;
            }
        }
        this.renderer.render(this.scene, this.camera);
    }
}
animate() {
    this.renderer.setAnimationLoop(this.render);
}

已尝试的实现 #

takePicture() {
    // 拍照
    this.screenshoot = this.renderer.domElement.toDataURL();
    console.log(this.screenshoot);
    alert("Work in Progress!");
},

解决方案 #

问题根源 #

WebXR会话中，Three.js的canvas仅负责渲染3D模型，移动端的真实AR背景由浏览器底层独立渲染，toDataURL()无法直接捕获这部分内容；同时移动端WebGL的帧缓冲机制可能导致截图时画布内容已被清空，出现空白。

修复步骤 #

1. 利用WebXR帧数据捕获完整画面
   直接从WebXR的XRFrame中读取包含AR背景的帧数据，而不是从Three.js的canvas提取：

   async takePicture() {
       const session = this.renderer.xr.getSession();
       if (!session) return;
   
       // 获取当前AR会话帧
       const frame = await new Promise(resolve => {
           session.requestAnimationFrame(resolve);
       });
       const glLayer = session.renderState.baseLayer;
       const canvas = glLayer.canvas;
       const gl = canvas.getContext('webgl');
   
       // 读取WebGL层像素数据
       const pixelBuffer = new Uint8Array(canvas.width * canvas.height * 4);
       gl.readPixels(0, 0, canvas.width, canvas.height, gl.RGBA, gl.UNSIGNED_BYTE, pixelBuffer);
   
       // 转换为可显示的图像（修正WebGL Y轴翻转问题）
       const captureCanvas = document.createElement('canvas');
       captureCanvas.width = canvas.width;
       captureCanvas.height = canvas.height;
       const ctx = captureCanvas.getContext('2d');
       const imageData = new ImageData(pixelBuffer, canvas.width, canvas.height);
       
       ctx.putImageData(imageData, 0, 0);
       ctx.scale(1, -1);
       ctx.drawImage(captureCanvas, 0, -canvas.height);
   
       // 生成截图并触发下载
       this.screenshoot = captureCanvas.toDataURL('image/png');
       const downloadLink = document.createElement('a');
       downloadLink.href = this.screenshoot;
       downloadLink.download = 'ar-snapshot.png';
       downloadLink.click();
   }
   

2. 确保配置兼容
   保持WebGLRenderer的preserveDrawingBuffer为true，同时确保WebXR会话的baseLayer正确关联到渲染画布。

3. 移动端适配
   调用拍照函数时必须确保AR会话处于活跃状态，且在动画帧回调内执行像素读取操作，避免帧缓冲被浏览器回收。

内容的提问来源于stack exchange，提问作者nazariisme