寻求Twilio短信、语音与Dialogflow CX集成的指导及代码排查

问题描述 #

我正在学习Twilio短信、语音与Dialogflow CX的集成，现有资料及文档不足且易混淆。已将Twilio号码与GCP Run构建的Webhook关联，但不确定是否遗漏关键步骤。参考了一个针对WhatsApp集成的仓库编写了代码，现寻求相关指导。

现有代码 #

imports...

const sessionClient = new SessionsClient({
  apiEndpoint: `${process.env.LOCATION}-dialogflow.googleapis.com`,
});

const app = express();

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

const listener = app.listen(process.env.PORT || 8081, () => {
  console.log('Your Dialogflow integration server is listening on port ' +
  listener.address().port);
});

const twilioToDetectIntent = (twilioReq) => {
    const sessionId = twilioReq.body.To;
    const sessionPath = sessionClient.projectLocationAgentSessionPath (
        process.env.PROJECT_ID,
        process.env.LOCATION,
        process.env.AGENT_ID,
        sessionId
    );

    const message = twilioReq.body.Body;
    const languageCode = process.env.LANGUAGE_CODE;
    const request = {
        session: sessionPath,
        queryInput:
            {
                text: {
                    text: message
                },
                languageCode
            }
        };
    
    return request;
};

const detectIntentToTwilio = (dialogflowResponse) => {
  let reply = "";
  
  for (let responseMessage of dialogflowResponse.queryResult.responseMessages) {
      if (responseMessage.hasOwnProperty('text')) {
          reply += responseMessage.text.text;
      }
  }

  const twiml = new  MessagingResponse();
  twiml.message(reply);
  return twiml;
};

const getResponseMessage = async (req) => {
  const dialogflowRequest = twilioToDetectIntent(req);
  const [dialogflowResponse] = await sessionClient.detectIntent(dialogflowRequest);
  const twiml = detectIntentToTwilio(dialogflowResponse);
  return twiml.toString();
};

//voice
const sendToDialogflowVoice = async (speechResult, callSid) => {
  const sessionPath = sessionClient.projectLocationAgentSessionPath(
    process.env.PROJECT_ID,
    process.env.LOCATION,
    process.env.AGENT_ID,
    callSid
  );

  const request = {
    session: sessionPath,
    queryInput: {
      audio: {
        config: {
          languageCode: process.env.LANGUAGE_CODE,
        },
        audio: speechResult, // El mensaje de voz
      },
    },
  };

  try {
    const [response] = await sessionClient.detectIntent(request);
    const messageFromDialogflow = response.queryResult.responseMessages[0].text.text;
    return messageFromDialogflow;
  } catch (error) {
    console.error('Something ocurred', error);
    return 'error from Dialogflow.';
  }
};

app.post('/sms', async (req, res) => {
  const message = await getResponseMessage(req);
    console.log("MESSAGE: " + message);
    res.send(message);
});

app.post('/voice', async (req, res) => {
  const speechResult = req.body.SpeechResult;
  const callSid = req.body.CallSid;
  const dialogflowResponse = await sendToDialogflowVoice(speechResult, callSid);

  const twiml = new MessagingResponse();
  twiml.message(dialogflowResponse);

  res.setHeader('Content-Type', 'text/xml');
  res.send(twiml.toString());
});

process.on('SIGTERM', () => {
  listener.close(async ()=> {
    console.log('Closing server.');
    process.exit(0);
  });
});

exports...

关键检查与优化建议 #

• 会话ID修正：短信模块当前用twilioReq.body.To（Twilio号码）作为会话ID，会导致所有用户共享对话上下文。应改为twilioReq.body.From（用户手机号），确保每个用户的会话独立：

  const sessionId = twilioReq.body.From;
  

• 语音TWIML类型错误：语音接口返回MessagingResponse是用于短信的，Twilio语音通话需要使用VoiceResponse并调用say方法：

  // 替换语音接口中的TWIML生成代码
  const twiml = new VoiceResponse();
  twiml.say(dialogflowResponse);
  

• 语音输入逻辑修正：sendToDialogflowVoice中把Twilio的SpeechResult（识别后的文本）作为audio参数传入是错误的。如果用Twilio转语音为文本，应改为文本输入：

  const request = {
    session: sessionPath,
    queryInput: {
      text: {
        text: speechResult
      },
      languageCode: process.env.LANGUAGE_CODE
    }
  };
  

  若要直接传递音频给Dialogflow，需配置Twilio发送RAW音频，再将音频数据转为Base64后传入audio字段。
• 异常处理增强：短信模块未处理detectIntent的异常，需添加try-catch：

  const getResponseMessage = async (req) => {
    try {
      const dialogflowRequest = twilioToDetectIntent(req);
      const [dialogflowResponse] = await sessionClient.detectIntent(dialogflowRequest);
      const twiml = detectIntentToTwilio(dialogflowResponse);
      return twiml.toString();
    } catch (error) {
      console.error('Dialogflow request failed:', error);
      const twiml = new MessagingResponse();
      twiml.message('抱歉，暂时无法回复，请稍后再试。');
      return twiml.toString();
    }
  };
  

• 响应消息拼接一致性：语音模块仅取第一个响应消息，应和短信模块一样遍历所有text类型消息拼接：

  let reply = "";
  for (let responseMessage of response.queryResult.responseMessages) {
    if (responseMessage.hasOwnProperty('text')) {
      reply += responseMessage.text.text.join(' ');
    }
  }
  return reply;
  

• 权限与配置验证：
  • 确认GCP Run环境变量PROJECT_ID、LOCATION、AGENT_ID、LANGUAGE_CODE已正确设置。
  • 确保Dialogflow CX Agent的服务账号拥有dialogflow.sessions.detectIntent权限，GCP Run服务账号拥有调用Dialogflow API的权限。
  • Twilio号码配置：短信Webhook指向/sms，方法POST，内容类型application/x-www-form-urlencoded；语音Webhook指向/voice，并启用语音识别（若用Twilio转文本）。

内容的提问来源于stack exchange，提问作者angelicacvo