安卓Chrome下第二深度相机调用readRenderTargetPixels高延迟求解

问题背景 #

我使用第二台深度相机渲染2x2分辨率的画面，仅需读取单像素数据，已尝试两种方案：

• Three.js的renderer.readRenderTargetPixels()（等价于原生WEBGL的gl.readPixels）
• 渲染到Canvas后调用getImageData()
  即便仅读取2x2像素渲染目标的内容，两种方案都会新增最多25ms的延迟，gl.readPixels效率低下的问题也已有公开技术讨论，需要可高效读取该单像素数据并写入JS数组或对象的解决方案。

Demo测试说明 #

我制作了演示readRenderTargetPixels()延迟问题的样例，使用说明如下：

1. 该demo同时支持普通模式和WebXR模式，无需进入VR即可观测延迟；JSFiddle存在移动端不显示VRButton的bug，如需在移动端进入VR，可将代码复制到index.html后通过WIFI运行HTTPS安全本地服务器，或使用支持显示VRButton的JSFiddle替代站点。
2. demo屏幕会展示最近10帧的最大、最小渲染耗时，普通模式和WebXR模式均支持，方便移动端查看。
3. 场景包含主相机和分辨率2x2、视场角0.015度的深度相机，深度相机对准旋转立方体测量深度，会在立方体表面绘制射线与点位，代码已做优化，去除了JS端高开销的解码运算。
4. 测试主要针对延迟更高的移动端，已提供二维码方便测试：扫描第一个二维码等待1分钟待性能稳定后，扫描第二个二维码做对比；如需查看编辑JSFiddle代码，将URL中的"show"删除后重载页面即可。
5. 请在Chrome浏览器下测试demo：PC端运行时请等待1分钟后再评估延迟，PC端火狐的延迟远低于Chrome但不支持所需功能，因此仅关注Chrome表现。测试中PC端Chrome初始渲染耗时约5ms（无该调用时仅1-2ms），后续耗时会增长2-3倍；高端安卓移动端耗时普遍在15-30ms区间。

已尝试方案&测试结果 #

• 使用Andre van Kammen的修改方案后，在小米Redmi Note 9S移动端仍存在极高延迟
• 尝试了像素缓冲对象（PBO）的异步GPU数据读取方案，该方案原本被认为可以避免GPU阻塞，但实际测试中开启和关闭该方案性能无任何差异。gl.readPixels推出至今已有30年，目前仍没有可靠高效的像素读取方案，希望能获得可行的优化建议。

演示代码 #

<!DOCTYPE html>
<html lang="en" >
<head>
    <title>READPIXELS LATENCY</title>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, viewport-fit=cover" />
    <meta name="mobile-web-app-capable" content="yes">
    <meta name="HandheldFriendly" content="true" />
    <style>
        html{height: 100%}
        body {text-align:center; background-color: #000000; height: 100%; margin: 0px; overflow: hidden}
        #info { position: absolute; top: 0px; width: 800px; margin: 10px; text-align: left; z-index: 90; display:none}
    </style>
</head>
<body>
    <div id="container"></div>
    <script type="module" defer>
        import * as THREE from "https://threejs.org/build/three.module.js";
        import { VRButton } from "https://threejs.org/examples/jsm/webxr/VRButton.js";

        var win = {
            x: window.innerWidth,
            y: window.innerHeight
        }

        const deg2rad = 0.017453292519943295769236907685;

        var tmp;
        var startms = 0;
        var endms = 0;
        var deltams = 0;
        var fcounter = 0;
        var fbuffer = [0,0,0,0,0,0,0,0,0,0];
        var maxms = 0;
        var minms = 1000;
        let avframes = 10;

        var scene = new THREE.Scene();
        scene.background = new THREE.Color( "#346189" );

        var xrRig = new THREE.Object3D();
        scene.add(xrRig);
        var fov = 50;
        var aspect = win.x/win.y;
        var cnear = 10;
        var cfar = 4000;
        var xrCam = new THREE.PerspectiveCamera( fov, aspect, cnear, cfar );
        xrRig.add(xrCam);
        xrRig.position.set(0, 20, 125);

        var depthRig = new THREE.Object3D();
        scene.add(depthRig);
        var dres = 2;
        var lfov = 0.015625;
        var laspect = 1;
        var lnear = 1;
        var lfar = 2000;
        var depthCam = new THREE.PerspectiveCamera( lfov, laspect, lnear, lfar );
        depthRig.add(depthCam);
        depthRig.position.set(40, 0, 50);
        depthRig.rotateOnAxis(new THREE.Vector3(0,1,0), 40 * deg2rad)

        var depthTarget = new THREE.WebGLRenderTarget( dres, dres );
        depthTarget.texture.format = THREE.RGBAFormat;

        var depthMaterial = new THREE.MeshDepthMaterial({depthPacking: THREE.RGBADepthPacking});
        var pb = new Uint8Array(4);
        var onpos = new THREE.Vector3();

        const Dlight = new THREE.DirectionalLight( 0xffffff, 1);
        Dlight.position.set( 0, 1000, 1000 );
        scene.add( Dlight );

        var renderer = new THREE.WebGLRenderer({ antialias: true, precision:'highp'});
        renderer.setPixelRatio( window.devicePixelRatio );
        renderer.setSize( win.x, win.y );
        renderer.autoClear = false;
        renderer.xr.enabled = true;

        var cont = document.getElementById( 'container' );
        cont.appendChild( renderer.domElement );
        document.body.appendChild( VRButton.createButton( renderer ) );

        var startray = new THREE.Vector3();
        var endray = new THREE.Vector3();
        var raygeom = new THREE.BufferGeometry();
        var points = [startray, endray];
        raygeom.setFromPoints( points );
        var rayline = new THREE.Line( raygeom, new THREE.MeshBasicMaterial({color: 0xff0000}) );
        scene.add(rayline);

        var marker = new THREE.Mesh(new THREE.SphereGeometry(0.8), new THREE.MeshBasicMaterial({color: 0xff0000}));
        scene.add(marker);

        var cubeGeometry = new THREE.BoxGeometry(40,40,40);
        var material = new THREE.MeshStandardMaterial({color: "#eabf11"});
        var cube = new THREE.Mesh(cubeGeometry, material);
        scene.add(cube);
        cube.position.set(0,0,0);

        var ccw = 500;
        var cch = 150;
        var vplane = new THREE.PlaneBufferGeometry( ccw, cch );
        var vmap = new THREE.MeshBasicMaterial();

        var m = new THREE.Mesh( vplane, vmap );
        m.visible = false;
        m.position.set(0, 150, -1000);
        scene.add( m );

        var canvas = document.createElement("canvas");
        var ctx = canvas.getContext("2d");
        ctx.canvas.width = ccw;
        ctx.canvas.height = cch;
        ctx.font = "60px Arial";
        ctx.textBaseline = "top";
        var img, tex;

        function drawtext(t1, t2){
            ctx.clearRect(0, 0, ccw, cch);
            ctx.fillStyle = "#346189";
            ctx.fillRect(0, 0, ccw, cch);
            ctx.fillStyle = "#ffffff";
            ctx.fillText(t1, 100, 10, ccw); 
            ctx.fillText(t2, 100, 80, ccw);     
            img = ctx.getImageData(0, 0, ccw, cch); 
            tex = new THREE.Texture(img);
            tex.needsUpdate = true;
            m.material.map = tex;
            m.material.needsUpdate = true;
            tex.dispose();
            m.visible = true;
        }

        window.addEventListener('resize', onResize, false);

        renderer.setAnimationLoop( render );

        function onResize(){
            if (renderer.xr.isPresenting) return;
            win.x = window.innerWidth;
            win.y = window.innerHeight;
            xrCam.aspect = win.x / win.y;
            xrCam.updateProjectionMatrix();
            renderer.setSize( win.x, win.y );
        }

        function render() {
            startms = Date.now();
            renderer.clear();
            cube.rotation.y += 0.01;
            
            renderer.xr.enabled = false;
            renderer.setClearColor("#ffffff", 1); 
            rayline.visible = false;
            marker.visible = false;
            renderer.setRenderTarget(depthTarget);
            scene.overrideMaterial = depthMaterial;
            renderer.render(scene, depthCam);

            renderer.readRenderTargetPixels(depthTarget, dres/2, dres/2, 1, 1, pb);

            var dp =  pb[0]*0.0000000002328306436538696 
                    + pb[1]*0.00000005960464477539063 
                    + pb[2]*0.0000152587890625 
                    + pb[3]*0.00390625;
            var viewZ = (lnear * lfar) / ((lfar - lnear) * dp - lfar);
            var midZ = viewZ;
            if (viewZ < -lfar) {
                midZ = -lfar;
            }
            onpos.set(0, 0, 0.5).applyMatrix4(depthCam.projectionMatrixInverse);
            onpos.multiplyScalar(midZ / onpos.z);
            onpos.applyMatrix4(depthCam.matrixWorld);
            startray = new THREE.Vector3();
            depthCam.getWorldPosition(startray);
            raygeom.attributes.position.setXYZ(0, startray.x, startray.y, startray.z);
            raygeom.attributes.position.setXYZ(1, onpos.x, onpos.y, onpos.z);
            raygeom.attributes.position.needsUpdate = true;

            renderer.setClearColor("#346189", 1); 
            renderer.xr.enabled = true;
            rayline.visible = true;
            marker.visible = true;
            marker.position.copy(onpos);

            scene.overrideMaterial = null;
            renderer.setRenderTarget(null);
            renderer.render( scene, xrCam ); 

            endms = Date.now();
            deltams = endms - startms;  
            for (let f=avframes; f>0; f--){
                tmp = fbuffer[f];
                minms = Math.min(tmp, minms);
                maxms = Math.max(tmp, maxms);
                fbuffer[f+1]=tmp;
            }
            fbuffer[1] = deltams;
            fcounter++;
            if (fcounter === avframes){
                fcounter = 0;
                drawtext("max-ms:"+maxms, "min-ms:"+minms);
                minms = 1000;
                maxms = 0;
            }
        }
    </script>
</body>
</html>

1. 调整读取时机避免GPU/CPU同步阻塞 #

当前你在渲染深度目标后立即调用readRenderTargetPixels，会强制CPU等待GPU完成当前帧所有渲染任务才能继续执行，这是延迟的核心来源。可调整为读取N-2帧的渲染结果：

• 维护2~3个循环复用的WebGLRenderTarget实例
• 第N帧渲染到目标A，不读取
• 第N+1帧渲染到目标B，读取目标A的像素
• 以此类推错开渲染和读取时机，Chrome会自动处理后台数据拷贝，不会阻塞主线程渲染，仅会有1~2帧的延迟，对于绝大多数非实时交互场景可接受

2. 修改像素格式降低传输开销 #

当前你使用RGBA8格式存储深度值，实际单通道精度足够的情况下可切换为RED格式（WebGL 2.0支持），仅传输1字节单像素数据，传输带宽直接降低75%，移动端收益尤其明显：

// 修改渲染目标配置
depthTarget.texture.format = THREE.RedFormat;
depthTarget.texture.type = THREE.UnsignedByteType;
// 读取时仅需要1字节的缓冲区
var pb = new Uint8Array(1);

3. 避免同一帧内多次切换渲染目标 #

你当前在单帧内先切到深度目标渲染、读取，再切回主画布渲染，多次上下文切换会带来额外开销。可调整渲染顺序：先完成主场景渲染，再渲染深度目标，下一帧再读取上一帧的深度数据，避免主渲染路径被打断。

4. 开启WebGL 2.0上下文以启用原生异步读取支持 #

可显式指定渲染器参数强制开启高性能模式：

var renderer = new THREE.WebGLRenderer({ antialias: true, precision:'highp', powerPreference: 'high-performance' });

WebGL 2.0下的PBO异步读取无效通常是因为调用时机不对，需要在gl.flush()之后的事件循环空闲阶段读取，不能紧跟在渲染指令之后调用，可以用requestIdleCallback包裹读取逻辑。

5. 替换深度读取方案为JS端射线检测 #

如果你的深度检测场景仅针对简单3D物体，可直接用Three.js内置的Raycaster做JS端射线检测，完全跳过GPU渲染和像素读取流程，单帧开销可以控制在1ms以内：

const raycaster = new THREE.Raycaster();
// 从深度相机位置发射正前方射线
raycaster.set(depthCam.getWorldPosition(new THREE.Vector3()), depthCam.getWorldDirection(new THREE.Vector3()));
// 检测场景内物体相交
const intersects = raycaster.intersectObjects(scene.children, true);
if (intersects.length > 0) {
    const onpos = intersects[0].point;
    // 后续点位处理逻辑和原有代码一致
}

该方案对于物体数量较少的场景性能远高于GPU读取像素的方案，且完全没有同步阻塞问题。

内容的提问来源于stack exchange，提问作者dllb