基于MS BotFramework NodeJS SDK自建多语言意图识别REST API问询

Awesome approach—building your own NLP solution makes total sense when LUIS’s multilingual capabilities aren’t meeting your bot’s needs. Let’s walk through how to integrate a custom neural intent classifier (exposed as a REST API) with your Node.js Bot Framework project, step by step:

First, define the core contract for your API to handle bilingual intent prediction:

• Request: Accepts a JSON payload with the user’s text and optionally a language tag (or auto-detect language internally). Example:

  {
    "text": "我想预订一张去北京的机票",
    "language": "zh"
  }
  

• Response: Returns the predicted intent and confidence score. Example:

  {
    "intent": "BookFlight",
    "confidence": 0.97
  }
  

For the neural model itself, use a multilingual pre-trained model like mBERT or XLM-RoBERTa—these are optimized for cross-language tasks and will handle both English and Chinese without needing separate models. Fine-tune the model on your labeled intent dataset (make sure to include both languages for each intent) using frameworks like PyTorch or TensorFlow.

Deploy this model as a REST API:

• If you’re comfortable with Python, use Flask or FastAPI to wrap your model endpoint.
• If you prefer staying in Node.js, use TensorFlow.js to run the model directly in a Node.js Express server.

In your Node.js bot, add logic to call your custom NLP API whenever a user sends a message. Here’s a practical example using axios for HTTP requests:

const axios = require('axios');
const { ActivityHandler } = require('botbuilder');

class BilingualBot extends ActivityHandler {
    constructor() {
        super();
        // Handle incoming messages
        this.onMessage(async (context, next) => {
            const userInput = context.activity.text;
            
            // Step 1: Detect language (optional—can also let your API handle this)
            const detectedLang = await this.detectLanguage(userInput);
            
            try {
                // Step 2: Call custom NLP API
                const nlpResponse = await axios.post('http://your-nlp-api-url/predict', {
                    text: userInput,
                    language: detectedLang
                });

                const { intent, confidence } = nlpResponse.data;

                // Step 3: Route conversation based on intent
                if (confidence < 0.7) {
                    await context.sendActivity("Sorry, I didn't catch that clearly. Could you rephrase?");
                } else {
                    switch(intent) {
                        case 'BookFlight':
                            await context.sendActivity(detectedLang === 'zh' ? '好的，我们开始预订机票。请问您从哪里出发？' : "Got it—let's start booking your flight. Where are you flying from?");
                            break;
                        case 'CheckBookingStatus':
                            await context.sendActivity(detectedLang === 'zh' ? '请提供您的预订编号，我来帮您查询状态。' : "Sure, can you share your booking reference so I can check the status?");
                            break;
                        // Add more intent handlers here
                        default:
                            await context.sendActivity(detectedLang === 'zh' ? '抱歉，我不太理解您的需求。' : "Sorry, I don't understand that request.");
                    }
                }
            } catch (error) {
                console.error('NLP API call failed:', error);
                await context.sendActivity(detectedLang === 'zh' ? '抱歉，暂时无法处理您的请求，请稍后再试。' : "Oops, something went wrong. Please try again later.");
            }

            await next();
        });
    }

    // Simple language detection using the `langdetect` package
    async detectLanguage(text) {
        const langDetect = require('langdetect');
        const detected = langDetect.detectOne(text);
        // Map to your API's supported language codes
        return detected === 'zh' ? 'zh' : 'en';
    }
}

module.exports.BilingualBot = BilingualBot;

• Dataset Quality: Ensure your training data has balanced examples for each intent in both English and Chinese. Include variations like slang, typos, and different sentence structures.
• Model Tuning: Fine-tune the multilingual model on your specific intent dataset—this will make predictions far more accurate than using a generic model.
• Latency Optimization: If your bot and NLP API are deployed separately, consider colocating them in the same cloud region to reduce network delay. For low-latency needs, you could even run the TensorFlow.js model directly in your bot’s Node.js server (no separate API needed).

• Low Confidence Fallback: As shown in the code example, if the intent confidence is below a threshold (e.g., 0.7), prompt the user to clarify their request.
• API Failure Fallback: Have a backup plan (like a simple rule-based matcher) for when the NLP API is unavailable.
• Multilingual Edge Cases: Handle code-switching (users mixing English and Chinese in one message) by training your model on such examples, or splitting the text and processing segments separately.

内容的提问来源于stack exchange，提问作者Ellery Leung