Google Dialogflow CX StreamingDetectIntent匹配首个意图后无法处理音频问题
环境详情
- 操作系统:Windows 10、11,Debian 9 (stretch)
- Node.js版本:12.18.3、12.22.1
- npm版本:7.19.0、7.15.0
@google-cloud/dialogflow-cx版本:2.13.0
问题描述
StreamingDetectIntent接口在匹配到首个意图后不再处理后续音频。可正常获取首次识别的转写结果,也能成功匹配到首个意图,但完成首次匹配后,音频仍在持续推流,却无法收到新的转写结果,on('data')回调也不会触发。简言之,首次意图匹配完成后无任何响应返回

目前可用的临时解决方案是:结束当前detectStream流后重新初始化,即可恢复正常工作。
复现步骤
分别尝试使用const {SessionsClient} = require("@google-cloud/dialogflow-cx");和const {SessionsClient} = require("@google-cloud/dialogflow-cx").v3;两种引入方式,结果一致。
// Create a stream for the streaming request. const detectStream = client .streamingDetectIntent() .on('error', console.error) .on('end', (data)=>{ console.log(`streamingDetectIntent: -----End-----: ${JSON.stringify(data)}`); }) .on('data', data => { console.log(`streamingDetectIntent: Data: ----------`); if (data.recognitionResult) { console.log(`Intermediate Transcript: ${data.recognitionResult.transcript}`); } else { console.log('Detected Intent:'); if(!data.detectIntentResponse) return const result = data.detectIntentResponse.queryResult; console.log(`User Query: ${result.transcript}`); for (const message of result.responseMessages) { if (message.text) { console.log(`Agent Response: ${message.text.text}`); } } if (result.match.intent) { console.log(`Matched Intent: ${result.match.intent.displayName}`); } console.log(`Current Page: ${result.currentPage.displayName}`); } }); const initialStreamRequest = { session: sessionPath, queryInput: { audio: { config: { audioEncoding: encoding, sampleRateHertz: sampleRateHertz, singleUtterance: true, }, }, languageCode: languageCode, } }; detectStream.write(initialStreamRequest);
分别尝试通过.wav音频文件推流、麦克风实时采集音频推流两种方式测试,均出现相同问题。
await pump( recordingStream, // microphone stream <OR> fs.createReadStream(audioFileName), // Format the audio stream into the request format. new Transform({ objectMode: true, transform: (obj, _, next) => { next(null, {queryInput: {audio: {audio: obj}}}); } }), detectStream );
参考了官方流识别示例实现以及RPC接口文档,未找到该问题的触发原因,特此求助。
内容的提问来源于stack exchange,提问作者Nishant Thapliyal
相关产品推荐
相关产品推荐

