使用Aforge与Affdex SDK多线程实现摄像头流及情绪识别故障咨询

Hey there, let's tackle this problem together. It's super common to run into thread conflicts when combining camera streaming libraries with emotion detection SDKs—so you're not alone here. Let's break down the likely issues and walk through actionable fixes to get both your camera feed and Affdex detection working properly.

1. Understand the Core Thread Conflicts #

The root issue here is probably thread safety and SDK thread affinity. Affdex SDK often expects its detection logic to run in a consistent, dedicated thread (not mixed with camera streaming threads or the UI thread). When you ran everything on the main thread, the camera stream was blocking the Affdex processing. When you moved the camera to a background thread, you might still be passing frames to Affdex incorrectly or not initializing the SDK in the right context.

2. Implement a Thread-Safe Frame Queue #

AForge's NewFrame event fires on the camera thread, and directly passing that frame to Affdex can cause resource locks or race conditions. Instead, use a thread-safe queue to pass frames from the camera thread to a dedicated Affdex processing thread.

Modified Code Example #

First, add the necessary fields:

using System.Collections.Concurrent;
using System.Threading;

private ConcurrentQueue<Bitmap> _frameQueue = new ConcurrentQueue<Bitmap>();
private Thread _affdexProcessingThread;
private bool _isDetectionRunning = true;

Update your camera start logic to spin up the Affdex thread:

public void camStart() 
{
    videoDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);
    videoSource = new VideoCaptureDevice(videoDevices[0].MonikerString);
    videoSource.NewFrame += video_NewFrame;
    CloseVideoSource();
    videoSource.Start();

    // Launch dedicated Affdex processing thread
    _affdexProcessingThread = new Thread(RunAffdexDetectionLoop);
    _affdexProcessingThread.Start();
}

Adjust the NewFrame handler to copy and queue frames (critical to avoid resource conflicts):

private void video_NewFrame(object sender, NewFrameEventArgs eventArgs)
{
    // Create a copy of the frame—don't use the original directly (it's locked by AForge)
    Bitmap frameCopy = new Bitmap(eventArgs.Frame);
    _frameQueue.Enqueue(frameCopy);
}

3. Dedicated Affdex Processing Thread #

Create a separate loop to handle all Affdex operations in one thread. This ensures the SDK's state stays consistent and avoids cross-thread issues.

private void RunAffdexDetectionLoop()
{
    // Initialize Affdex detector HERE (in the same thread that will process frames)
    var frameDetector = new Affdex.FrameDetector();
    
    // Configure your detection settings
    frameDetector.SetDetectAllEmotions(true);
    frameDetector.FaceDetectorMode = Affdex.FaceDetectorMode.LARGE_FACES; // Adjust as needed
    
    // Start the detector
    frameDetector.Start();

    while (_isDetectionRunning)
    {
        if (_frameQueue.TryDequeue(out Bitmap currentFrame))
        {
            try
            {
                // Process the frame with Affdex
                frameDetector.Process(currentFrame, DateTime.Now.Ticks);

                // Retrieve and handle detection results
                foreach (var face in frameDetector.Faces)
                {
                    // Example: Access emotion values
                    float joyScore = face.Emotions.Joy;
                    float angerScore = face.Emotions.Anger;

                    // If updating UI, use Invoke (WinForms) or Dispatcher (WPF) to cross threads
                    // this.Invoke(new Action(() => {
                    //     lblJoy.Text = $"Joy: {joyScore:P0}";
                    // }));
                }
            }
            catch (Exception ex)
            {
                // Log errors to debug what's failing
                Console.WriteLine($"Affdex detection error: {ex.Message}");
            }
            finally
            {
                // Always dispose the frame to prevent memory leaks
                currentFrame.Dispose();
            }
        }
        else
        {
            // Short sleep to avoid spiking CPU when queue is empty
            Thread.Sleep(10);
        }
    }

    // Clean up Affdex resources properly
    frameDetector.Stop();
    frameDetector.Dispose();
}

4. Proper Cleanup Logic #

Make sure to stop both threads safely when closing the camera:

private void CloseVideoSource()
{
    // Signal Affdex thread to stop
    _isDetectionRunning = false;
    _affdexProcessingThread?.Join(5000); // Wait up to 5 seconds for cleanup

    // Stop camera stream
    if (videoSource != null && videoSource.IsRunning)
    {
        videoSource.SignalToStop();
        videoSource.WaitForStop();
        videoSource.Dispose();
    }
}

5. Additional Troubleshooting Tips #

• Check Affdex Logs: Enable SDK logging to see if there are specific errors (e.g., unsupported frame format, initialization failures). Most SDKs have a way to enable verbose logging.
• Frame Format Compatibility: Ensure the camera's output format (e.g., RGB24) is supported by Affdex. If not, convert the frame to a compatible format before passing it to the detector.
• Thread Priorities: Adjust thread priorities if needed—give the Affdex thread a slightly higher priority than the camera thread to ensure detection doesn't get starved.
• SDK Initialization: Never initialize the Affdex detector on one thread and use it on another. All detector operations (init, process, stop) must happen in the same thread.

内容的提问来源于stack exchange，提问作者Rana Wael