如何在React中同时录制屏幕与摄像头视频并合并为单个视频？

核心思路 #

不用保留两个独立录制的视频流，而是通过Canvas实时合成屏幕流与摄像头流，再录制Canvas的输出流，最终直接得到单文件合成视频（屏幕为主画面，摄像头画面以小窗口形式叠加）。

实现步骤与代码示例 #

1. 组件实现（React） #

import { useState, useRef, useEffect } from 'react';

const ScreenCamCombinedRecorder = () => {
  const canvasRef = useRef(null);
  const screenStreamRef = useRef(null);
  const camStreamRef = useRef(null);
  const mediaRecorderRef = useRef(null);
  const [isRecording, setIsRecording] = useState(false);
  const [recordedVideoBlob, setRecordedVideoBlob] = useState(null);

  // 获取屏幕与摄像头媒体流
  const fetchMediaStreams = async () => {
    // 捕获屏幕共享流（含音频可选）
    const screenStream = await navigator.mediaDevices.getDisplayMedia({
      video: true,
      audio: true
    });
    // 捕获摄像头流（小窗口尺寸）
    const camStream = await navigator.mediaDevices.getUserMedia({
      video: { width: 320, height: 240 },
      audio: false // 避免音频重复，复用屏幕音频
    });

    screenStreamRef.current = screenStream;
    camStreamRef.current = camStream;
    startCanvasRenderLoop();
  };

  // 实时将双流渲染到Canvas
  const startCanvasRenderLoop = () => {
    const canvas = canvasRef.current;
    const ctx = canvas.getContext('2d');
    const screenTrack = screenStreamRef.current.getVideoTracks()[0];
    
    // 同步Canvas尺寸与屏幕分辨率
    const trackSettings = screenTrack.getSettings();
    canvas.width = trackSettings.width;
    canvas.height = trackSettings.height;

    // 创建隐藏的视频元素用于获取流帧
    const screenVideo = document.createElement('video');
    screenVideo.srcObject = screenStreamRef.current;
    screenVideo.play();
    const camVideo = document.createElement('video');
    camVideo.srcObject = camStreamRef.current;
    camVideo.play();

    // 循环绘制帧
    const renderFrame = () => {
      if (!isRecording) return;
      // 绘制全屏屏幕流
      ctx.drawImage(screenVideo, 0, 0, canvas.width, canvas.height);
      // 绘制右下角摄像头小窗口（距边缘20px）
      const camWidth = 320;
      const camHeight = 240;
      ctx.drawImage(
        camVideo,
        canvas.width - camWidth - 20,
        canvas.height - camHeight - 20,
        camWidth,
        camHeight
      );
      requestAnimationFrame(renderFrame);
    };
    renderFrame();
  };

  // 开始录制合成流
  const startRecording = () => {
    setIsRecording(true);
    // 获取Canvas的捕获流（30fps）
    const canvasStream = canvasRef.current.captureStream(30);
    // 添加屏幕音频流
    if (screenStreamRef.current.getAudioTracks().length > 0) {
      canvasStream.addTrack(screenStreamRef.current.getAudioTracks()[0]);
    }
    // 初始化MediaRecorder
    mediaRecorderRef.current = new MediaRecorder(canvasStream, {
      mimeType: 'video/webm; codecs=vp9'
    });
    const chunks = [];
    mediaRecorderRef.current.ondataavailable = (e) => chunks.push(e.data);
    mediaRecorderRef.current.onstop = () => {
      const combinedBlob = new Blob(chunks, { type: 'video/webm' });
      setRecordedVideoBlob(combinedBlob);
    };
    mediaRecorderRef.current.start();
  };

  // 停止录制并释放资源
  const stopRecording = () => {
    setIsRecording(false);
    mediaRecorderRef.current.stop();
    // 停止所有媒体轨道
    screenStreamRef.current.getTracks().forEach(track => track.stop());
    camStreamRef.current.getTracks().forEach(track => track.stop());
  };

  // 播放合成后的视频
  const playCombinedVideo = () => {
    if (recordedVideoBlob) {
      const videoUrl = URL.createObjectURL(recordedVideoBlob);
      window.open(videoUrl);
    }
  };

  return (
    <div>
      {/* 隐藏Canvas，仅用于后台合成 */}
      <canvas ref={canvasRef} style={{ display: 'none' }} />
      <button onClick={fetchMediaStreams} disabled={isRecording}>获取媒体流</button>
      <button onClick={startRecording} disabled={!screenStreamRef.current || isRecording}>开始录制</button>
      <button onClick={stopRecording} disabled={!isRecording}>停止录制</button>
      <button onClick={playCombinedVideo} disabled={!recordedVideoBlob}>播放合成视频</button>
    </div>
  );
};

export default ScreenCamCombinedRecorder;

2. 关键细节说明 #

• Canvas尺寸同步：通过屏幕流的轨道参数获取真实分辨率，避免画面拉伸变形。
• 音频处理：仅保留屏幕流的音频，避免双音频叠加产生杂音。
• 格式选择：video/webm; codecs=vp9具备较好的浏览器兼容性与压缩率，若需MP4格式，可尝试video/mp4（部分现代浏览器支持）。
• 性能优化：使用requestAnimationFrame确保渲染帧率与系统同步，减少卡顿。

3. 后处理合并方案（已录制双视频文件） #

如果已经保存了独立的屏幕和摄像头视频文件，可通过FFmpeg命令行完成合并：

ffmpeg -i screen_recording.mp4 -i cam_recording.mp4 -filter_complex "[0:v][1:v]overlay=W-w-20:H-h-20[v]" -map "[v]" -map 0:a combined_output.mp4

该命令会将摄像头视频叠加到屏幕视频的右下角，距离边缘20像素。

内容的提问来源于stack exchange，提问作者javed iqbal