如何从Darknet(C)传递视频流至Dlib(C++)实现目标跟踪？

Hey there! Let's walk through how to integrate Darknet's real-time object detection with Dlib's tracking by passing IplImage frames from demo.c to Dlib. No extra "C/C++ connectors" are needed—since C++ is compatible with C, we can directly mix the two codebases with a few tweaks.

Step 1: Prepare Darknet's demo code for C++ #

Since Dlib is a C++ library, we need to compile Darknet's demo code as C++:

• Rename demo.c to demo.cpp (this tells the compiler to use C++ rules).
• Wrap Darknet's C headers in extern "C" to prevent name mangling (C++ modifies function names differently than C):

  extern "C" {
      #include "darknet.h"
      #include "image.h"
      // Add other Darknet headers you need here
  }
  

Step 2: Convert IplImage to Dlib's image format #

Dlib works best with its own image types, but it has built-in support for OpenCV's IplImage via the dlib/opencv.h header. Here's how to convert frames:

#include <dlib/opencv.h>
#include <dlib/image_processing.h>
#include <dlib/gui_widgets.h>

// Later, in your frame processing loop:
IplImage* current_frame = ...; // Get this from Darknet's video capture code

// Wrap the IplImage in a Dlib cv_image (matches BGR format of IplImage)
dlib::cv_image<dlib::bgr_pixel> dlib_bgr_frame(current_frame);

// Optional: Convert to RGB if your tracker expects it (most Dlib tools work with both)
dlib::array2d<dlib::rgb_pixel> dlib_rgb_frame;
dlib::convert_image(dlib_bgr_frame, dlib_rgb_frame);

Step 3: Integrate Dlib's tracking into Darknet's pipeline #

1. Initialize the tracker(s)
   Add this near the top of your main function (or initialization section):

   // For single object tracking
   dlib::correlation_tracker tracker;
   bool tracker_initialized = false;
   

   For multi-object tracking, use a std::vector<dlib::correlation_tracker> to track multiple targets.

2. Initialize tracker with Darknet's detection results
   After Darknet runs detection on a frame (look for where detections are processed in demo.cpp), use the bounding box to start tracking:

   // Assume detection is a valid Darknet detection struct with bbox.x, bbox.y, bbox.w, bbox.h
   if (!tracker_initialized && detection->confidence > 0.5) { // Use your confidence threshold
       dlib::rectangle track_bbox(
           detection->bbox.x,
           detection->bbox.y,
           detection->bbox.x + detection->bbox.w,
           detection->bbox.y + detection->bbox.h
       );
       tracker.start_track(dlib_rgb_frame, track_bbox);
       tracker_initialized = true;
   }
   

3. Update tracking on subsequent frames
   In every frame after initialization, update the tracker and get the current bounding box:

   if (tracker_initialized) {
       tracker.update(dlib_rgb_frame);
       dlib::rectangle current_track_bbox = tracker.get_position();
   
       // Draw the tracking box on the original IplImage (using OpenCV)
       cvRectangle(
           current_frame,
           cvPoint(current_track_bbox.left(), current_track_bbox.top()),
           cvPoint(current_track_bbox.right(), current_track_bbox.bottom()),
           CV_RGB(0, 255, 0), // Green color for tracking box
           2
       );
   }
   

Step 4: Compile and Link #

Modify Darknet's Makefile to support C++:

• Change CC = gcc to CC = g++ (or ensure demo.cpp is compiled with g++).
• Add Dlib's include path and library link flags. For example:

  CFLAGS += -I/path/to/dlib/include
  LDFLAGS += -L/path/to/dlib/build -ldlib -std=c++11
  

  Make sure Dlib is compiled with the same compiler and flags as your project.

Key Notes #

• Multi-object tracking: Extend the single tracker setup to a vector of trackers, one per detected target. Re-initialize trackers if Darknet detects new objects or existing ones drift too far.
• Performance: For real-time use, consider running detection less frequently (e.g., every 5 frames) and using tracking in between to reduce compute load.
• Color format: Double-check that your frame format matches—IplImage is typically BGR, so dlib::cv_image<dlib::bgr_pixel> is the correct wrapper.

内容的提问来源于stack exchange，提问作者Jay Singh