You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Azure通信服务Web端群组通话是否支持内置批注功能?

Azure Communication Services 群组通话批注功能问题

我反复查阅了Azure Communication Services文档,未找到JavaScript SDK支持群组通话批注功能的相关说明。

目前我通过FabricJS捕获远程视频流,在其上添加画布并绘制,再将合并后的视频流伪装成本地摄像头画面发送,但存在诸多问题:发送给远端参与者的流压缩严重,宽高比/缩放异常,对方仅能看到批注视频的中心区域。

我想了解微软是否提供官方的批注添加方式,或是否有更优的替代方案/库?以下是相关代码:

async annotate() {
    try {
      // Get the video element
      const videoElement = this.technicianVideoContainer.querySelector('video');
      if (!videoElement) {
        console.error('No video element found');
        return;
      }

      // Create a canvas container element
      const canvasContainer = document.createElement('div');
      canvasContainer.className = 'canvas-container';
      canvasContainer.style.width = `${videoElement.clientWidth}px`;
      canvasContainer.style.height = `${videoElement.clientHeight}px`;
      canvasContainer.style.position = 'absolute';
      canvasContainer.style.left = `${videoElement.offsetLeft}px`;
      canvasContainer.style.top = `${videoElement.offsetTop}px`;
      canvasContainer.style.objectFit = `none`;
      canvasContainer.style.zIndex = '100';

      // Append the canvas container to the body or a suitable container
      this.technicianVideoContainer.appendChild(canvasContainer);

      // Create a canvas element inside the canvas container
      this.canvas = new fabric.Canvas(document.createElement('canvas'));
      this.canvas.setWidth(videoElement.clientWidth); // Match video element width
      this.canvas.setHeight(videoElement.clientHeight); // Match video element height

      // Append the canvas to the canvas container
      canvasContainer.appendChild(this.canvas.wrapperEl);

      // Set the z-index of the canvas to be higher than the video element
      this.canvas.wrapperEl.style.zIndex = '101';

      // Add the video element to the canvas as an image
      const videoImage = new fabric.Image(videoElement, {
        left: 0,
        top: 0,
        angle: 0,
        selectable: false // Make it non-selectable for now as drawing will happen later
      });
      this.canvas.add(videoImage); // Add the video image to the canvas

      // Update the canvas with the video frame on a regular interval
      setInterval(() => {
        if (this.canvas) {
          this.canvas.requestRenderAll(); // Re-render the canvas to update the video frame
        }
      }, 100);

      await this.toggleDrawingMode();
    } catch (error) {
      console.error('Error annotating video:', error);
    }
  }



  async toggleDrawingMode() {
    if (!this.canvas) {
      console.error('Canvas is not initialized.');
      return;
    }
  
    this.canvas.isDrawingMode = !this.canvas.isDrawingMode;
    this.canvas.freeDrawingBrush.color = 'red';
    this.canvas.freeDrawingBrush.width = 2;
  
    if (this.canvas.isDrawingMode) {
      // Enable drawing mode
      this.canvas.on('mouse:down', (options) => {
        if (options && options.e) {
          const pointer = this.canvas.getPointer(options.e);
          const eventOptions = { e: options.e };
          this.canvas.freeDrawingBrush.onMouseDown(pointer, eventOptions);
          this.canvas.renderAll();
        }
      });
  
      this.canvas.on('mouse:up', async (options) => {
        if (options && options.e) {
          const remoteVideoStream = this.remoteVideoStreams.find(stream => stream.isAvailable);
          if (!remoteVideoStream) {
            console.error('No available remote video stream found');
            return;
          }
  
          const technicianVideoStream = await remoteVideoStream.getMediaStream();
          const videoElement = document.createElement('video');
          videoElement.srcObject = technicianVideoStream;
          videoElement.play();
  
          // Create a new canvas to combine video and annotations
          const compositeCanvas = document.createElement('canvas');
          compositeCanvas.width = this.canvas.getWidth();
          compositeCanvas.height = this.canvas.getHeight();
          const ctx = compositeCanvas.getContext('2d');
  
          const drawFrame = () => {
            ctx.clearRect(0, 0, compositeCanvas.width, compositeCanvas.height);
            ctx.drawImage(videoElement, 0, 0, compositeCanvas.width, compositeCanvas.height);
  
            // Render the fabric.js canvas onto the composite canvas
            this.canvas.renderAll();
  
            // Draw the fabric.js canvas onto the composite canvas
            ctx.drawImage(this.canvas.getElement(), 0, 0);
  
            requestAnimationFrame(drawFrame);
          };
  
          videoElement.addEventListener('play', () => {
            drawFrame();
          });
  
          // Capture the combined stream
          const combinedStream = compositeCanvas.captureStream(60); // 30 FPS
          // let a =combinedStream.getVideoTracks()[0]
          const combinedLocalVideoStream = new LocalVideoStream(combinedStream);
  
          // const videoStreamRenderer = new VideoStreamRenderer(combinedLocalVideoStream);
          // const view = await videoStreamRenderer.createView();
          // this.technicianVideoContainer.appendChild(view.target);
  
          // Start sending the combined video stream in the call
          if (!this.localVideoOn) {
            await this.call.startVideo(combinedLocalVideoStream);
            this.localVideoOn = true;
          }

        }
      });
  
    } else {
      // Disable drawing mode
      this.canvas.off('mouse:down');
      this.canvas.off('mouse:up');
    }
  }
subscribeToRemoteVideoStream = async (remoteVideoStream: RemoteVideoStream, remoteParticipantUserId: string) => {
    let renderer = new VideoStreamRenderer(remoteVideoStream);
    let view;
    this.remoteVideoContainer = document.createElement('div');
    this.remoteVideoContainer.className = 'remote-video-container';

    const videoContainerStyle = 'height: 100%; border: solid';
    this.remoteVideoContainer.setAttribute('style', videoContainerStyle);
    let loadingSpinner = document.createElement('div');
    loadingSpinner.className = 'loading-spinner';
    const isUserTechnician = this.videoData.usersAndTokens[0].user.id === remoteParticipantUserId;

    remoteVideoStream.on('isReceivingChanged', async () => {
      try {
        console.log("video changed")
          if (remoteVideoStream.isAvailable) {
            if (this.canvas) {
              await this.stopVideo();
              this.destroyCanvas();
              await this.annotate();
            }
              this.remoteVideoStreams = this.remoteVideoStreams.concat(remoteVideoStream);
              const isReceiving = remoteVideoStream.isReceiving;
              const isLoadingSpinnerActive = this.remoteVideoContainer.contains(loadingSpinner);
              if (!isReceiving && !isLoadingSpinnerActive) {
                  this.remoteVideoContainer.appendChild(loadingSpinner);
              } else if (isReceiving && isLoadingSpinnerActive) {
                  this.remoteVideoContainer.removeChild(loadingSpinner);
              }
              this.remoteVideoOn = true;
          } else {
              this.remoteVideoOn = false;
          }
      } catch (e) {
          console.error(e);
      }
  });
}

组件其余部分及HTML遵循Azure Communication Services官方快速入门示例代码。

我尝试过FabricJS和Konva:FabricJS能实现批注,但远端接收的视频压缩严重、画面异常;Konva的实现逻辑与FabricJS接近,但远端无法接收到视频。这些方案均不理想,因此希望获取微软官方的批注实现方式,或更优的替代方案,而非通过伪装本地视频流的方式实现。


内容的提问来源于stack exchange,提问作者Gal Zant

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.06.18 20:24:52