如何用aiortc替代视频文件，通过Mss实现桌面实时视频流？

你当前的代码依赖MediaPlayer处理本地视频文件，要实现桌面实时流，需要用mss捕获屏幕帧，并自定义MediaStreamTrack适配aiortc的视频传输要求。以下是完整的解决方案：

步骤1：安装依赖 #

确保已安装所需库：

pip install mss aiortc aiohttp opencv-python numpy

步骤2：替换代码实现桌面捕获 #

修改原代码，替换视频源逻辑，添加自定义屏幕捕获轨道类：

import asyncio
import json
import logging
import os
import numpy as np
import cv2
from mss import mss
from aiohttp import web
from aiortc import RTCPeerConnection, RTCSessionDescription, MediaStreamTrack

ROOT = os.path.dirname(__file__)

class ScreenShareTrack(MediaStreamTrack):
    kind = "video"

    def __init__(self):
        super().__init__()
        self.sct = mss()
        # 捕获主显示器，可通过monitor参数指定其他显示器或自定义区域
        self.monitor = self.sct.monitors[1]

    async def recv(self):
        # 捕获屏幕帧
        frame = self.sct.grab(self.monitor)
        # 将mss的RGBA格式转为BGR
        img = np.array(frame)
        img = cv2.cvtColor(img, cv2.COLOR_RGBA2BGR)
        # 转为WebRTC标准的YUV420p格式
        img_yuv = cv2.cvtColor(img, cv2.COLOR_BGR2YUV_I420)
        # 构建aiortc所需的VideoFrame
        from aiortc import VideoFrame
        return VideoFrame.from_ndarray(img_yuv, format="yuv420p")

async def index(request):
    content = open(os.path.join(ROOT, "index.html"), "r").read()
    return web.Response(content_type="text/html", text=content)

async def offer(request):
    params = await request.json()
    offer = RTCSessionDescription(sdp=params["sdp"], type=params["type"])
    pc = RTCPeerConnection()
    pcs.add(pc)

    # 替换为屏幕捕获轨道
    video = ScreenShareTrack()
    pc.addTrack(video)

    await pc.setRemoteDescription(offer)
    answer = await pc.createAnswer()
    await pc.setLocalDescription(answer)

    return web.Response(
        content_type="application/json",
        text=json.dumps(
            {"sdp": pc.localDescription.sdp, "type": pc.localDescription.type}
        ),
    )

pcs = set()
async def on_shutdown(app):
    coros = [pc.close() for pc in pcs]
    await asyncio.gather(*coros)
    pcs.clear()

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    app = web.Application()
    app.on_shutdown.append(on_shutdown)
    app.router.add_get("/", index)
    app.router.add_post("/offer", offer)
    web.run_app(app, host="0.0.0.0", port=8080)

关键说明 #

• ScreenShareTrack类继承MediaStreamTrack，通过recv()方法持续向WebRTC提供屏幕帧
• 若要捕获特定显示器或自定义区域，可修改self.monitor参数，例如：self.monitor = {"top": 0, "left": 0, "width": 1920, "height": 1080}
• 利用opencv完成图像格式转换，将mss捕获的RGBA帧转为WebRTC支持的YUV420p格式，确保客户端正常解码

内容的提问来源于stack exchange，提问作者Deve loper