Azure通信服务Web端群组通话是否支持内置批注功能?
Azure Communication Services 群组通话批注功能问题
我反复查阅了Azure Communication Services文档,未找到JavaScript SDK支持群组通话批注功能的相关说明。
目前我通过FabricJS捕获远程视频流,在其上添加画布并绘制,再将合并后的视频流伪装成本地摄像头画面发送,但存在诸多问题:发送给远端参与者的流压缩严重,宽高比/缩放异常,对方仅能看到批注视频的中心区域。
我想了解微软是否提供官方的批注添加方式,或是否有更优的替代方案/库?以下是相关代码:
async annotate() { try { // Get the video element const videoElement = this.technicianVideoContainer.querySelector('video'); if (!videoElement) { console.error('No video element found'); return; } // Create a canvas container element const canvasContainer = document.createElement('div'); canvasContainer.className = 'canvas-container'; canvasContainer.style.width = `${videoElement.clientWidth}px`; canvasContainer.style.height = `${videoElement.clientHeight}px`; canvasContainer.style.position = 'absolute'; canvasContainer.style.left = `${videoElement.offsetLeft}px`; canvasContainer.style.top = `${videoElement.offsetTop}px`; canvasContainer.style.objectFit = `none`; canvasContainer.style.zIndex = '100'; // Append the canvas container to the body or a suitable container this.technicianVideoContainer.appendChild(canvasContainer); // Create a canvas element inside the canvas container this.canvas = new fabric.Canvas(document.createElement('canvas')); this.canvas.setWidth(videoElement.clientWidth); // Match video element width this.canvas.setHeight(videoElement.clientHeight); // Match video element height // Append the canvas to the canvas container canvasContainer.appendChild(this.canvas.wrapperEl); // Set the z-index of the canvas to be higher than the video element this.canvas.wrapperEl.style.zIndex = '101'; // Add the video element to the canvas as an image const videoImage = new fabric.Image(videoElement, { left: 0, top: 0, angle: 0, selectable: false // Make it non-selectable for now as drawing will happen later }); this.canvas.add(videoImage); // Add the video image to the canvas // Update the canvas with the video frame on a regular interval setInterval(() => { if (this.canvas) { this.canvas.requestRenderAll(); // Re-render the canvas to update the video frame } }, 100); await this.toggleDrawingMode(); } catch (error) { console.error('Error annotating video:', error); } } async toggleDrawingMode() { if (!this.canvas) { console.error('Canvas is not initialized.'); return; } this.canvas.isDrawingMode = !this.canvas.isDrawingMode; this.canvas.freeDrawingBrush.color = 'red'; this.canvas.freeDrawingBrush.width = 2; if (this.canvas.isDrawingMode) { // Enable drawing mode this.canvas.on('mouse:down', (options) => { if (options && options.e) { const pointer = this.canvas.getPointer(options.e); const eventOptions = { e: options.e }; this.canvas.freeDrawingBrush.onMouseDown(pointer, eventOptions); this.canvas.renderAll(); } }); this.canvas.on('mouse:up', async (options) => { if (options && options.e) { const remoteVideoStream = this.remoteVideoStreams.find(stream => stream.isAvailable); if (!remoteVideoStream) { console.error('No available remote video stream found'); return; } const technicianVideoStream = await remoteVideoStream.getMediaStream(); const videoElement = document.createElement('video'); videoElement.srcObject = technicianVideoStream; videoElement.play(); // Create a new canvas to combine video and annotations const compositeCanvas = document.createElement('canvas'); compositeCanvas.width = this.canvas.getWidth(); compositeCanvas.height = this.canvas.getHeight(); const ctx = compositeCanvas.getContext('2d'); const drawFrame = () => { ctx.clearRect(0, 0, compositeCanvas.width, compositeCanvas.height); ctx.drawImage(videoElement, 0, 0, compositeCanvas.width, compositeCanvas.height); // Render the fabric.js canvas onto the composite canvas this.canvas.renderAll(); // Draw the fabric.js canvas onto the composite canvas ctx.drawImage(this.canvas.getElement(), 0, 0); requestAnimationFrame(drawFrame); }; videoElement.addEventListener('play', () => { drawFrame(); }); // Capture the combined stream const combinedStream = compositeCanvas.captureStream(60); // 30 FPS // let a =combinedStream.getVideoTracks()[0] const combinedLocalVideoStream = new LocalVideoStream(combinedStream); // const videoStreamRenderer = new VideoStreamRenderer(combinedLocalVideoStream); // const view = await videoStreamRenderer.createView(); // this.technicianVideoContainer.appendChild(view.target); // Start sending the combined video stream in the call if (!this.localVideoOn) { await this.call.startVideo(combinedLocalVideoStream); this.localVideoOn = true; } } }); } else { // Disable drawing mode this.canvas.off('mouse:down'); this.canvas.off('mouse:up'); } }
subscribeToRemoteVideoStream = async (remoteVideoStream: RemoteVideoStream, remoteParticipantUserId: string) => { let renderer = new VideoStreamRenderer(remoteVideoStream); let view; this.remoteVideoContainer = document.createElement('div'); this.remoteVideoContainer.className = 'remote-video-container'; const videoContainerStyle = 'height: 100%; border: solid'; this.remoteVideoContainer.setAttribute('style', videoContainerStyle); let loadingSpinner = document.createElement('div'); loadingSpinner.className = 'loading-spinner'; const isUserTechnician = this.videoData.usersAndTokens[0].user.id === remoteParticipantUserId; remoteVideoStream.on('isReceivingChanged', async () => { try { console.log("video changed") if (remoteVideoStream.isAvailable) { if (this.canvas) { await this.stopVideo(); this.destroyCanvas(); await this.annotate(); } this.remoteVideoStreams = this.remoteVideoStreams.concat(remoteVideoStream); const isReceiving = remoteVideoStream.isReceiving; const isLoadingSpinnerActive = this.remoteVideoContainer.contains(loadingSpinner); if (!isReceiving && !isLoadingSpinnerActive) { this.remoteVideoContainer.appendChild(loadingSpinner); } else if (isReceiving && isLoadingSpinnerActive) { this.remoteVideoContainer.removeChild(loadingSpinner); } this.remoteVideoOn = true; } else { this.remoteVideoOn = false; } } catch (e) { console.error(e); } }); }
组件其余部分及HTML遵循Azure Communication Services官方快速入门示例代码。
我尝试过FabricJS和Konva:FabricJS能实现批注,但远端接收的视频压缩严重、画面异常;Konva的实现逻辑与FabricJS接近,但远端无法接收到视频。这些方案均不理想,因此希望获取微软官方的批注实现方式,或更优的替代方案,而非通过伪装本地视频流的方式实现。
内容的提问来源于stack exchange,提问作者Gal Zant
相关产品推荐
相关产品推荐

