如何用Watson Unity SDK创建自定义语音转文本模型提升城市识别准确率？

Hey there! I’ve tackled exactly this problem before with Watson’s Speech-to-Text in Unity, so let’s walk through how to build a custom model, train it, and integrate it into your existing flight booking app—all with C# scripts.

1. Create & Train a Custom Language Model (IBM Cloud Console) #

First, you’ll need to set up your custom model directly in the IBM Cloud to teach Watson your target city names and context. Here’s how:

• Log into your IBM Cloud account, navigate to your Speech-to-Text service instance, and go to the Custom models tab.
• Click Create custom model: Pick the language matching your app (e.g., en-US), give it a name like "Flight City Names", and add a brief description.
• Once the model is created, go to its Words section. Add each problematic city name (Berlin, Paris, London, etc.)—you can even add pronunciation variants to cover different accents. For example:
  • Word: London, Pronunciations: LUN-dun, LON-dun
  • Word: Berlin, Pronunciations: bur-LIN, BER-lin
• Optional but highly recommended: Upload sample transcriptions of your app’s actual voice interactions (e.g., "I want a flight to Paris") as training data. This helps Watson learn the context around your city names.
• Click Train model and wait for it to finish (this can take anywhere from a few minutes to an hour depending on data size). Once it’s marked "Available", copy its Customization ID—you’ll need this for Unity.

2. Integrate the Custom Model into Unity C# #

Assuming you already have the IBM Watson Unity SDK imported into your project, here’s how to modify your Speech-to-Text script to use the custom model:

Example Speech-to-Text Manager Script #

using IBM.Watson.SpeechToText.V1;
using IBM.Watson.SpeechToText.V1.Model;
using UnityEngine;

public class WatsonSTTController : MonoBehaviour
{
    [Header("Watson Credentials")]
    [SerializeField] private string sttApiKey;
    [SerializeField] private string sttServiceUrl;
    [SerializeField] private string customModelId; // Paste your Customization ID here

    private SpeechToTextService _speechToText;
    private RecognizeStream _activeRecognitionStream;

    void Start()
    {
        InitializeSpeechToText();
    }

    private void InitializeSpeechToText()
    {
        // Set up the STT service with your credentials
        _speechToText = new SpeechToTextService();
        _speechToText.SetCredential(sttApiKey, sttServiceUrl);

        // Configure recognition to use your custom model
        var recognitionOptions = new RecognizeOptions
        {
            Model = "en-US_BroadbandModel", // Match your custom model's language
            CustomizationId = customModelId, // Critical: Enables your custom model
            InterimResults = true, // Optional: Get partial recognition results
            MaxAlternatives = 1,
            SmartFormatting = true // Optional: Clean up capitalization/punctuation
        };

        // Initialize the WebSocket stream for real-time recognition
        _activeRecognitionStream = _speechToText.RecognizeUsingWebSocket(
            OnRecognitionResult, 
            OnRecognitionError, 
            recognitionOptions
        );
    }

    // Callback for successful recognition results
    private void OnRecognitionResult(SpeechRecognitionEvent result)
    {
        if (result?.results == null || result.results.Length == 0) return;

        foreach (var resultSegment in result.results)
        {
            if (resultSegment.final) // Only process final, non-interim results
            {
                string recognizedText = resultSegment.alternatives[0].transcript.Trim();
                Debug.Log($"Recognized Text: {recognizedText}");
                
                // Pass the result to your existing flight query logic
                var flightManager = FindObjectOfType<FlightQueryHandler>();
                if (flightManager != null)
                {
                    flightManager.ProcessUserQuery(recognizedText);
                }
            }
        }
    }

    // Callback for recognition errors
    private void OnRecognitionError(Error error)
    {
        Debug.LogError($"Speech-to-Text Error: {error?.Message ?? "Unknown error"}");
    }

    // Public methods to control listening (call these from UI buttons or your interaction script)
    public void StartListening()
    {
        if (_activeRecognitionStream != null && !_activeRecognitionStream.IsListening)
        {
            _activeRecognitionStream.StartListening();
            Debug.Log("Started listening...");
        }
    }

    public void StopListening()
    {
        if (_activeRecognitionStream != null && _activeRecognitionStream.IsListening)
        {
            _activeRecognitionStream.StopListening();
            Debug.Log("Stopped listening.");
        }
    }
}

3. Integrate with Your Existing Interaction Script #

Let’s say your existing flight query script is called FlightQueryHandler—you just need to make sure it has a public method to accept the recognized text. For example:

using UnityEngine;

public class FlightQueryHandler : MonoBehaviour
{
    // Reference your Watson Assistant and TTS components here
    [SerializeField] private WatsonAssistantController assistantController;

    public void ProcessUserQuery(string userInput)
    {
        // Pass the recognized text to Watson Assistant for intent parsing
        assistantController.SendMessageToAssistant(userInput);
    }

    // Rest of your existing logic (handling assistant responses, TTS, etc.)
}

4. Additional Tips for Better Accuracy #

• Update the custom model regularly: If you notice new misrecognized cities, add them to the model’s word list and retrain.
• Use context phrases: Add flight-related terms like "departure from", "arrival in", "flight to" to your custom model’s word list—this helps Watson understand the context of the city names.
• Test with real users: Have users with different accents test your app, then refine the pronunciation variants in the custom model based on their input.

内容的提问来源于stack exchange，提问作者grunter-hokage