Mapbox GL动画与H264编码同线程卡顿的多线程优化方案咨询

我首次在StackOverflow提问，目前遇到棘手问题：Mapbox动画与H264-MP4视频编码器在同一线程运行，代码功能正常，但动画渲染时出现轻微跳帧甚至抖动，确定是双任务同线程导致的性能问题。我尝试将编码器移入Worker，但因编码器依赖Mapbox实例报错；也了解过OffscreenCanvas，但WebGL相关资料较少。希望能获得将编码器或Mapbox移至Worker的可行方案，或是更优的解决思路。

主线程相关代码 #

map.on("load", async () => {
        // add 3d, sky and fog
        add3D();
        await map.once("idle");
        // don't forget to enable WebAssembly SIMD in chrome://flags for faster encoding
        const supportsSIMD = await simd();
        // initialize H264 video encoder
        const Encoder = await loadEncoder({ simd: supportsSIMD });
        const gl = map.painter.context.gl;
        const width =
          gl.drawingBufferWidth % 2 === 0
            ? gl.drawingBufferWidth
            : gl.drawingBufferWidth - 1;
        const height =
          gl.drawingBufferHeight % 2 === 0
            ? gl.drawingBufferHeight
            : gl.drawingBufferHeight - 1;

        //create encoder
        const encoder = Encoder.create({
          width,
          height,
          fps: 30,
          kbps: 2200,
          rgbFlipY: true,
        });
        // stub performance.now for deterministic rendering per-frame (only available in dev build)
        let now = performance.now();
        //mapboxgl.setNow(now);
        const ptr = encoder.getRGBPointer(); // keep a pointer to encoder WebAssembly heap memory
        function frame() { //this function is called to read the pixels every time mapbox re-renders
          //increment stub time by 16.6ms (60 fps)
          now += 1000 / 30;
          mapboxgl.setNow(now);
          const pixels = encoder.memory().subarray(ptr); // get a view into encoder memory
          gl.readPixels(0, 0, width, height, gl.RGBA, gl.UNSIGNED_BYTE, pixels); // read pixels into encoder
          console.log("inicia");
          encoder.encodeRGBPointer(); // encode the frame
        }
        map.on("render", frame); // set up frame-by-frame

        await playAnimations(geojson);//a call is made to the map animation function
        // stop recording
        map.off("render", frame);
        mapboxgl.restoreNow();
        setIsRunAnimate(false);
        //encoded video
        const mp4Blob = encoder.end();
        setVideoBlob(mp4Blob);
})

尝试的Worker代码（因依赖mapboxgl报错） #

// encoderWorker.js

let encoding = false;
let now = performance.now();
let ptr;

onmessage = async (event) => {
  const { width, height, fps, kbps, rgbFlipY } = event.data;

  const { simd } = await import("wasm-feature-detect");
  const Encoder = await import("../../assets/convertidor.js");

  const encoder = Encoder.create({
    width,
    height,
    fps,
    kbps,
    rgbFlipY,
  });

  ptr = encoder.getRGBPointer();

  function encodeFrame() {
    now += 1000 / fps;
    mapboxgl.setNow(now);
    const pixels = encoder.memory().subarray(ptr);
    encoder.encodeRGBPointer();
  }

  onmessage = (event) => {
    const { type } = event.data;
    if (type === "start" && !encoding) {
      // Start coding/frame-by-frame
      encoding = true;
      encodeFrame();
      requestAnimationFrame(encodeFrame);
    } else if (type === "finish" && encoding) {
      // Stop encryption
      encoding = false;

      // Return the encoded blob to the main thread
      const encodedBlob = encoder.end();
      postMessage({ type: "complete", encodedBlob });
    }
  };
};

可行优化方案 #

方案1：异步化编码任务，避免阻塞主线程渲染 #

不要在render事件中同步执行编码逻辑，将编码任务延迟到主线程空闲时执行：

function frame() {
  now += 1000 / 30;
  mapboxgl.setNow(now);
  const pixels = encoder.memory().subarray(ptr);
  gl.readPixels(0, 0, width, height, gl.RGBA, gl.UNSIGNED_BYTE, pixels);
  // 用requestIdleCallback让主线程优先完成渲染
  requestIdleCallback(() => {
    encoder.encodeRGBPointer();
  });
}

也可以用setTimeout将编码推入任务队列，保证渲染先完成：

setTimeout(() => encoder.encodeRGBPointer(), 0);

方案2：正确分离编码器到Worker，传递像素数据而非依赖Mapbox实例 #

Worker无法直接访问主线程的Mapbox实例，需在主线程读取像素后，将数据传递给Worker完成编码：

1. 主线程修改：

// 初始化Worker
const encoderWorker = new Worker('./encoderWorker.js');
// 传递编码器初始化参数
encoderWorker.postMessage({
  type: 'init',
  width,
  height,
  fps:30,
  kbps:2200,
  rgbFlipY:true
});

function frame() {
  now += 1000 / 30;
  mapboxgl.setNow(now);
  const pixels = new Uint8Array(gl.drawingBufferWidth * gl.drawingBufferHeight *4);
  gl.readPixels(0, 0, width, height, gl.RGBA, gl.UNSIGNED_BYTE, pixels);
  // 转移像素缓冲区所有权，避免数据复制
  encoderWorker.postMessage({
    type: 'encode',
    pixels
  }, [pixels.buffer]);
}

// 监听Worker编码完成事件
encoderWorker.onmessage = (e) => {
  if(e.data.type === 'complete') {
    setVideoBlob(e.data.encodedBlob);
  }
};

// 动画结束后通知Worker停止编码
await playAnimations(geojson);
encoderWorker.postMessage({type: 'finish'});

2. Worker代码修改：

let encoder = null;

onmessage = async (event) => {
  const { type } = event.data;
  if(type === 'init') {
    const { width, height, fps, kbps, rgbFlipY } = event.data;
    const { simd } = await import("wasm-feature-detect");
    const EncoderModule = await import("../../assets/convertidor.js");
    const supportsSIMD = await simd();
    const Encoder = await EncoderModule.loadEncoder({ simd: supportsSIMD });
    encoder = Encoder.create({
      width,
      height,
      fps,
      kbps,
      rgbFlipY,
    });
  } else if(type === 'encode' && encoder) {
    const { pixels } = event.data;
    const ptr = encoder.getRGBPointer();
    const encoderMem = encoder.memory().subarray(ptr, ptr + pixels.length);
    encoderMem.set(pixels);
    encoder.encodeRGBPointer();
  } else if(type === 'finish' && encoder) {
    const mp4Blob = encoder.end();
    postMessage({ type: 'complete', encodedBlob: mp4Blob });
    encoder = null;
  }
};

方案3：使用OffscreenCanvas分离Mapbox渲染到Worker（实验性） #

Mapbox部分版本支持OffscreenCanvas，可尝试将渲染逻辑移入Worker：

• 在主线程创建OffscreenCanvas，关联到页面的Canvas元素
• 将OffscreenCanvas传递给Worker，在Worker内初始化Mapbox实例
• 注意：需确认当前Mapbox版本支持Worker内的WebGL上下文，且浏览器支持OffscreenCanvas特性

内容的提问来源于stack exchange，提问作者palomino