OpenCV C++ CSRT跟踪器初始化报错求助：merge函数断言失败

Let's break down why you're hitting that merge assertion error when initializing the CSRT tracker, and fix the float-to-Mat conversion issue along the way.

What's Causing the Error? #

The error (-215:Assertion failed) mv[i].size == mv[0].size && mv[i].depth() == depth in function 'merge' typically pops up when OpenCV expects an image of a specific depth/channel format but gets something incompatible.

Looking at your code:

• You're creating a CV_32FC4 Mat from the float* RGBA data, scaling it to 0-1 range, then converting to BGR. The result is a 32-bit floating-point BGR image (CV_32FC3).
• OpenCV's CSRT tracker (like most OpenCV tracking algorithms) requires input images to be 8-bit unsigned (CV_8UC3 for color, CV_8UC1 for grayscale). The 32-bit float format isn't supported here, which triggers the internal merge function failure during tracker initialization.

Step-by-Step Fixes #

1. Convert the 32-bit Float Image to 8-bit Unsigned #

After converting to BGR, you need to scale the 0-1 float values back to 0-255 and convert to 8-bit format. Add this line right after cvtColor:

cv::Mat first_frame_8u;
first_frame.convertTo(first_frame_8u, CV_8UC3, 255.0);

• The 255.0 multiplier scales your 0-1 float values back to the 0-255 range expected for 8-bit images.
• CV_8UC3 tells OpenCV to create a 3-channel, 8-bit unsigned Mat.

2. Use the 8-bit Image for Tracker Initialization #

Update your tracker init call to use the new 8-bit image instead of the float one:

tracker->init(first_frame_8u, bbox);

You should also use this 8-bit image for displaying with imshow (since imshow handles 8-bit images best):

rectangle(first_frame_8u, bbox, Scalar(255, 0, 0), 2, 1);
imshow("Tracking", first_frame_8u);

Full Corrected Code Snippet #

Here's how your code should look after the fixes:

cv::Mat first_frame;
float* first_imgRGBA = NULL;
camera->CaptureRGBA(&first_imgRGBA, 1000, 1);

// Create 32-bit float RGBA Mat
first_frame = cv::Mat(camera->GetHeight(), camera->GetWidth(), CV_32FC4, first_imgRGBA);
first_frame /= 255; // Scale to 0-1 range

// Convert RGBA to BGR (still 32-bit float)
cv::cvtColor(first_frame, first_frame, cv::COLOR_RGBA2BGR);

// Convert to 8-bit unsigned BGR (required for CSRT tracker)
cv::Mat first_frame_8u;
first_frame.convertTo(first_frame_8u, CV_8UC3, 255.0);

// Initialize tracker and display
Rect2d bbox(287, 23, 86, 320);
rectangle(first_frame_8u, bbox, Scalar(255, 0, 0), 2, 1);
imshow("Tracking", first_frame_8u);
tracker->init(first_frame_8u, bbox);

Additional Checks #

• Verify that your camera's float* output is indeed in the 0-255 range (since you're dividing by 255). If the camera outputs 0-1 float values directly, skip the first_frame /= 255 line and adjust the convertTo multiplier to 1.0 (but this is less common for RGBA camera outputs).
• Ensure your bbox coordinates are within the bounds of the image (width/height from camera->GetWidth()/GetHeight()). Out-of-bounds boxes can cause unexpected errors too, though your current error points to format issues first.

内容的提问来源于stack exchange，提问作者efedoganay