如何在Azure中接收电话并流式传输至认知服务意图识别

一、核心逻辑 #

要完成来电接收并流式传输到认知服务（CLU），需要结合Azure通信服务（ACS）的Call Automation和媒体流推送能力：通过Event Grid接收来电事件并接听通话，再通过WebSocket获取ACS推送的实时音频流，最后将流转发给认知服务做意图识别；也可以直接让ACS对接认知服务完成语音转文本，再将文本传给CLU处理。

二、分步实现 #

1. 启用ACS媒体流功能 #

先在ACS资源后台配置媒体流推送，指定你的WebSocket端点（即代码中wss://...地址），通话接通后ACS会自动将音频流通过该WebSocket推送到你的服务。

2. 处理来电事件并接听通话 #

你现有的Event Grid处理逻辑可以保留，修正接听代码确保媒体流配置生效：

[AllowAnonymous]
[HttpPost]
public async Task<IActionResult> IncomingCall([FromBody] EventGridEvent[] events)
{
    foreach (var eventGridEvent in events)
    {
        switch (eventGridEvent.EventType)
        {
            case SystemEventNames.EventGridSubscriptionValidation:
                var eventData = eventGridEvent.Data.ToObjectFromJson<SubscriptionValidationEventData>();
                return Ok(new SubscriptionValidationResponse { ValidationResponse = eventData.ValidationCode });
            case "Microsoft.Communication.IncomingCall":
                var _client = new CallAutomationClient("<ACS连接字符串>");
                var eventData2 = eventGridEvent.Data.ToObjectFromJson<AcsIncomingCallEventData>();
                string incomingCallContext = eventData2.IncomingCallContext;
                
                // 接听时指定WebSocket端点接收媒体流，可选配置认知服务端点直接做语音转文本
                var answerCallOptions = new AnswerCallOptions(incomingCallContext, new Uri("wss://你的WebSocket服务地址"))
                {
                    AzureCognitiveServicesEndpointUrl = new Uri("https://你的认知服务资源地址.cognitiveservices.azure.com/")
                };
                
                var call = await _client.AnswerCallAsync(answerCallOptions);
                break;
        }
    }
    return Ok();
}

3. 接收WebSocket音频流并转发到CLU #

实现WebSocket服务接收ACS推送的音频流，再将流传给认知服务的Intent Recognizer做实时意图识别（ACS推送的音频为PCM 16kHz/16位/单声道，与认知服务格式兼容）：

public async Task HandleWebSocketConnection(WebSocket webSocket)
{
    var buffer = new byte[1024 * 4];
    WebSocketReceiveResult result;

    // 初始化认知服务配置
    var speechConfig = SpeechConfig.FromSubscription("<语音服务密钥>", "<语音服务区域>");
    speechConfig.SpeechRecognitionLanguage = "en-US";
    speechConfig.SetProperty(PropertyId.Speech_SegmentationSilenceTimeoutMs, "2000");
    
    // 绑定CLU意图模型
    var languageModel = LanguageUnderstandingModel.FromAppId("<CLU应用ID>");
    using var intentRecognizer = new IntentRecognizer(speechConfig);
    intentRecognizer.AddAllIntents(languageModel);

    // 实时识别回调
    intentRecognizer.Recognizing += (s, e) =>
    {
        if (e.Result.Reason == ResultReason.RecognizingIntent)
        {
            // 处理实时识别中的意图和文本
            Console.WriteLine($"识别中：意图={e.Result.IntentId}，文本={e.Result.Text}");
        }
    };

    intentRecognizer.Recognized += (s, e) =>
    {
        if (e.Result.Reason == ResultReason.RecognizedIntent)
        {
            // 处理最终识别结果
            Console.WriteLine($"最终结果：意图={e.Result.IntentId}，置信度={e.Result.Intents[0].ConfidenceScore}");
        }
    };

    // 持续接收WebSocket音频流并推送至认知服务
    using var pushStream = AudioInputStream.CreatePushStream();
    using var audioConfig = AudioConfig.FromStreamInput(pushStream);
    await intentRecognizer.StartContinuousRecognitionAsync();

    while ((result = await webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None)).Count > 0)
    {
        if (result.MessageType == WebSocketMessageType.Binary)
        {
            pushStream.Write(buffer, 0, result.Count);
        }
        else if (result.MessageType == WebSocketMessageType.Close)
        {
            await webSocket.CloseAsync(WebSocketCloseStatus.NormalClosure, string.Empty, CancellationToken.None);
            break;
        }
    }

    await intentRecognizer.StopContinuousRecognitionAsync();
}

4. 简化方案：ACS直接对接认知服务转文本后传CLU #

如果不需要自己处理音频流，可以让ACS完成语音转文本，再通过Event Grid将文本推送给你的服务，最后传给CLU：

// 接听时指定认知服务端点
var answerCallOptions = new AnswerCallOptions(incomingCallContext, new Uri("https://你的回调接口地址"))
{
    AzureCognitiveServicesEndpointUrl = new Uri("https://你的认知服务资源地址.cognitiveservices.azure.com/")
};
var answerCallResult = await callAutomationClient.AnswerCallAsync(answerCallOptions);

// 回调接口接收转文本结果并传给CLU
[HttpPost]
public async Task<IActionResult> Callback([FromBody] EventGridEvent[] events)
{
    foreach (var evt in events)
    {
        if (evt.EventType == "Microsoft.Communication.TranscriptionCompleted")
        {
            var transcriptionData = evt.Data.ToObjectFromJson<AcsTranscriptionCompletedEventData>();
            string transcribedText = transcriptionData.Transcription;
            
            // 将文本传给CLU识别意图
            var speechConfig = SpeechConfig.FromSubscription("<语音服务密钥>", "<语音服务区域>");
            using var intentRecognizer = new IntentRecognizer(speechConfig);
            var languageModel = LanguageUnderstandingModel.FromAppId("<CLU应用ID>");
            intentRecognizer.AddAllIntents(languageModel);
            
            var result = await intentRecognizer.RecognizeOnceAsync();
            if (result.Reason == ResultReason.RecognizedIntent)
            {
                // 处理意图识别结果
            }
        }
    }
    return Ok();
}

三、关键注意事项 #

• 确保WebSocket服务稳定，避免连接断开导致音频流丢失；
• 提前在Azure认知服务中训练好CLU意图模型，获取对应的应用ID和密钥；
• ACS推送的音频格式无需额外转换，直接兼容认知服务Speech SDK。

内容的提问来源于stack exchange，提问作者Carl Erik Lindberg