基于Unity Render Streaming与WebRTC实现Unity向JS发文本消息的问题

我正在用Unity搭建流媒体应用，基于Unity Render Streaming插件结合RTCPeerConnection实现视频流传输。修改了双向WebApp示例的sendvideo.js，仅向Unity客户端发送视频，用ReceiverSample.cs渲染视频。现在需要在视频流传输时从Unity向JS发送"Hello world"这类文本消息，但尝试失败，且ReceiverSample.cs出现编译错误。

当前环境 #

• WebRTC版本：2.3.3-preview
• Unity Render Streaming版本：3.0.1-preview
• Unity版本：2022.3.45f1

尝试的代码 #

JavaScript端（sendvideo.js） #

import * as Logger from "../../module/logger.js";

export class SendVideo {
  constructor(localVideoElement, remoteVideoElement) {
    this.localVideo = localVideoElement;
    this.remoteVideo = remoteVideoElement;
    this.peerConnection = null;
    this.dataChannel = null; // Add DataChannel variable
  }

  async startLocalVideo() {
    try {
      const videoElement = document.createElement('video');
      videoElement.src = '/videos/video.mp4'; // Path to your video file
      videoElement.muted = true;
      await videoElement.play();

      const stream = videoElement.captureStream();
      this.localVideo.srcObject = stream;
      await this.localVideo.play();

      this.initializePeerConnection();

    } catch (err) {
      Logger.error(`Error starting local video: ${err}`);
    }
  }

  // Set up WebRTC connection and DataChannel
  initializePeerConnection() {
    this.peerConnection = new RTCPeerConnection();

    this.dataChannel = this.peerConnection.createDataChannel("myDataChannel");

    this.dataChannel.onopen = () => {
      console.log("DataChannel open: Connection established with Unity client");
    };

    this.dataChannel.onmessage = (event) => {
      console.log(`Received message from Unity: ${event.data}`); 
    };

    this.peerConnection.createOffer().then(offer => {
      return this.peerConnection.setLocalDescription(offer);
    }).then(() => {
      console.log("Offer created and sent to Unity client");
    }).catch(err => console.error(err));
  }
}

C#端（ReceiverSample.cs） #

using System;
using System.Collections;
using UnityEngine;
using UnityEngine.UI;
using Unity.WebRTC; 

public class ReceiverSample : MonoBehaviour
{
#pragma warning disable 0649
    [SerializeField] private Button startButton;
    [SerializeField] private Button stopButton;
    [SerializeField] private InputField connectionIdInput;
    [SerializeField] private RawImage remoteVideoImage;
    [SerializeField] private ReceiveVideoViewer receiveVideoViewer;
    [SerializeField] private SingleConnection connection;
#pragma warning restore 0649

    private RTCDataChannel dataChannel;
    private float messageInterval = 2.0f; // Send a message every 2 seconds

    void Awake()
    {
        startButton.onClick.AddListener(OnStart);
        stopButton.onClick.AddListener(OnStop);
        if (connectionIdInput != null)
            connectionIdInput.onValueChanged.AddListener(input => connectionId = input);
        receiveVideoViewer.OnUpdateReceiveTexture += texture => remoteVideoImage.texture = texture;
    }

    private void OnStart()
    {
        if (string.IsNullOrEmpty(connectionId))
        {
            connectionId = System.Guid.NewGuid().ToString("N");
            connectionIdInput.text = connectionId;
        }
        connectionIdInput.interactable = false;

        // Create the connection
        connection.CreateConnection(connectionId, true);

        // Subscribe to the DataChannel event (available from the connection object)
        connection.OnDataChannel += OnDataChannelReceived;

        startButton.gameObject.SetActive(false);
        stopButton.gameObject.SetActive(true);
    }

    private void OnDataChannelReceived(RTCDataChannel channel)
    {
        dataChannel = channel;

        dataChannel.OnOpen += () =>
        {
            Debug.Log("DataChannel opened. Starting to send messages.");
            StartCoroutine(SendHelloWorldMessage());
        };

        dataChannel.OnClose += () =>
        {
            Debug.Log("DataChannel closed.");
        };

        dataChannel.OnMessage += OnMessageReceived;
    }

    private void OnMessageReceived(byte[] message)
    {
        string receivedMessage = System.Text.Encoding.UTF8.GetString(message);
        Debug.Log($"Message received from WebApp: {receivedMessage}");
    }

    private IEnumerator SendHelloWorldMessage()
    {
        while (dataChannel.ReadyState == RTCDataChannelState.Open)
        {
            dataChannel.Send("Hello World");
            Debug.Log("Sent 'Hello World' to JavaScript client");
            yield return new WaitForSeconds(messageInterval);
        }
    }

    private void OnStop()
    {
        StopAllCoroutines(); 
        connection.DeleteConnection(connectionId);
        connectionId = String.Empty;
        connectionIdInput.text = String.Empty;
        connectionIdInput.interactable = true;
        startButton.gameObject.SetActive(true);
        stopButton.gameObject.SetActive(false);
    }
}

编译错误 #

Assets\Samples\Unity Render Streaming\3.0.1-preview\Example\Receiver\ReceiverSample.cs(15,30): error CS0246: The type or namespace name 'SingleConnection' could not be found (are you missing a using directive or an assembly reference?)

我想确认RTCDataChannel方案是否适用于我的场景，若可行，该如何修复错误使其正常工作。

1. RTCDataChannel方案的适用性 #

完全适用。WebRTC的RTCDataChannel就是为在已建立的PeerConnection链路之上，同步传输文本/二进制数据设计的，刚好匹配你在视频流传输同时发送消息的需求，无需额外的通信链路，性能和兼容性都有保障。

2. 修复编译错误（CS0246） #

这个错误是因为SingleConnection是Unity Render Streaming示例中的自定义组件类，不属于WebRTC或Render Streaming的核心API，修复步骤如下：

• 检查并导入示例资源：确认项目中存在SingleConnection.cs文件（通常位于Assets/Samples/Unity Render Streaming/3.0.1-preview/Example/Common目录下），如果没有，重新导入Unity Render Streaming的完整示例包。
• 添加命名空间引用：在ReceiverSample.cs的顶部添加示例类的命名空间引用：

  using Unity.RenderStreaming.Samples;
  

• 绑定组件实例：在Unity编辑器的Inspector面板中，将场景中的SingleConnection组件拖入ReceiverSample的connection字段中，确保引用有效。

3. 确保消息正常收发的额外调整 #

除了修复编译错误，还需要完善信令和连接逻辑，否则DataChannel无法正常建立：

JS端完善信令交互 #

当前sendvideo.js仅创建了Offer，但没有完成WebRTC必需的信令交换流程，需要补充：

• 连接到Unity Render Streaming默认的WebSocket信令服务器
• 将本地Offer发送给Unity客户端
• 接收Unity返回的Answer并设置到PeerConnection
• 交换ICE候选地址
  示例信令逻辑片段（参考官方WebApp示例）：

// 在initializePeerConnection中添加信令相关逻辑
this.peerConnection.onicecandidate = (event) => {
  if (event.candidate) {
    // 发送ICE候选到信令服务器，转发给Unity
    sendSignalingMessage({ type: 'ice', candidate: event.candidate });
  }
};

// 监听信令服务器的消息，处理Unity的Answer和ICE候选
signalingSocket.onmessage = (event) => {
  const message = JSON.parse(event.data);
  switch(message.type) {
    case 'answer':
      this.peerConnection.setRemoteDescription(new RTCSessionDescription(message.answer));
      break;
    case 'ice':
      this.peerConnection.addIceCandidate(new RTCIceCandidate(message.candidate));
      break;
  }
};

Unity端的细节检查 #

• 确保ReceiveVideoViewer和SingleConnection组件已正确配置在场景中
• 测试时使用localhost或HTTPS协议（浏览器对HTTP下的WebRTC有安全限制）
• 检查Unity Console中的日志，确认DataChannel的OnOpen事件是否触发，若未触发则说明PeerConnection未成功建立

内容的提问来源于stack exchange，提问作者Rick James