如何在React这类ES6 Node.js环境中运行MediaPipe FaceMesh

故障原因 #

• 你在onResults回调中操作的是自行创建的未挂载到DOM的canvas元素，绘制的内容没有渲染到页面可见的canvas节点上
• FaceMesh、camera、canvasCtx等实例没有用useRef持久化存储，组件重渲染时会被重复创建导致状态丢失
• 没有配置本地的模型资源路径，仍在调用远程CDN资源，离线或者网络波动时会加载失败
• useEffect的依赖逻辑不合理，camera实例会被重复初始化

修复方案 #

1. 替换CDN资源路径为本地npm包资源 #

Create React App环境下webpack会自动处理打包资源，直接修改FaceMesh的locateFile配置即可：

const faceMesh = new FaceMesh({
  locateFile: (file) => {
    return require(`@mediapipe/face_mesh/${file}`);
  }
});

2. 修正canvas引用 #

删除代码中const canvasElement = document.createElement('canvas');这行，所有canvas相关操作都使用你在JSX中声明的canvasReference.current实例：

function onResults(results) {
  if(!canvasReference.current || !canvasCtx.current) return
  const canvas = canvasReference.current
  const ctx = canvasCtx.current
  ctx.save();
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  ctx.drawImage(results.image, 0, 0, canvas.width, canvas.height);
  // 原有绘制逻辑保持不变，统一使用ctx进行绘制
  if (results.multiFaceLandmarks) {
    for (const landmarks of results.multiFaceLandmarks) {
      drawConnectors(ctx, landmarks, FACEMESH_TESSELATION, { color: '#C0C0C070', lineWidth: 1 });
      drawConnectors(ctx, landmarks, FACEMESH_RIGHT_EYE, { color: '#FF3030' });
      drawConnectors(ctx, landmarks, FACEMESH_RIGHT_EYEBROW, { color: '#FF3030' });
      drawConnectors(ctx, landmarks, FACEMESH_LEFT_EYE, { color: '#30FF30' });
      drawConnectors(ctx, landmarks, FACEMESH_LEFT_EYEBROW, { color: '#30FF30' });
      drawConnectors(ctx, landmarks, FACEMESH_FACE_OVAL, { color: '#E0E0E0' });
      drawConnectors(ctx, landmarks, FACEMESH_LIPS, { color: '#E0E0E0' });
    }
  }
  ctx.restore();
}

3. 持久化实例避免重渲染丢失 #

把所有需要跨渲染周期保留的变量都用useRef存储：

const webcamRef = React.useRef(null);
const canvasReference = React.useRef(null);
const canvasCtx = React.useRef(null);
const camera = React.useRef(null);
const faceMesh = React.useRef(null);
const [cameraReady, setCameraReady] = useState(false);

4. 调整初始化逻辑 #

将FaceMesh和Camera的初始化放在同一个useEffect中，确保只执行一次，同时添加组件销毁时的资源释放逻辑：

useEffect(() => {
  // 等待摄像头和canvas挂载完成再初始化
  if(!webcamRef.current || !canvasReference.current || !cameraReady) return
  // 初始化canvas上下文
  canvasCtx.current = canvasReference.current.getContext('2d')
  // 手动设置canvas的实际像素尺寸，避免绘制变形
  canvasReference.current.width = 1280
  canvasReference.current.height = 720
  // 初始化FaceMesh
  faceMesh.current = new FaceMesh({
    locateFile: (file) => require(`@mediapipe/face_mesh/${file}`)
  })
  faceMesh.current.setOptions({
    selfieMode: true,
    maxNumFaces: 1,
    minDetectionConfidence: 0.5,
    minTrackingConfidence: 0.5
  })
  faceMesh.current.onResults(onResults)
  // 初始化摄像头
  camera.current = new Camera(webcamRef.current.video, {
    onFrame: async () => {
      await faceMesh.current.send({ image: webcamRef.current.video })
    },
    width: 1280,
    height: 720
  })
  camera.current.start()
  // 组件销毁时释放资源
  return () => {
    camera.current?.stop()
    faceMesh.current?.close()
  }
}, [cameraReady])

内容的提问来源于stack exchange，提问作者Lorenzo SU