Android端Haar Cascade检测后调用Optical flow跟踪报错如何解决

OpenCV(3.4.12) Error: Assertion failed ((npoints = prevPtsMat.checkVector(2, CV_32F, true)) >= 0) in virtual void cv::{anonymous}::SparsePyrLKOpticalFlowImpl::calc(cv::InputArray, cv::InputArray, cv::InputArray, cv::InputOutputArray, cv::OutputArray, cv::OutputArray), file /build/3_4_pack-android/opencv/modules/video/src/lkpyramid.cpp, line 1259

根因定位 #

calcOpticalFlowPyrLK接口对输入的前序关键点矩阵有严格格式要求：必须是2通道、CV_32F浮点类型的点集。你的代码存在两个触发报错的问题：

• previousPoints类型错误：你当前使用的是MatOfPoint类型（存储int型坐标），不符合接口要求的MatOfPoint2f类型（存储float型坐标），格式校验直接失败
• 空点集无校验：当Haar级联检测未检出任何目标时，CurrentCars为空列表，赋值后的previousPoints为空，同样不满足断言要求

修复步骤 #

1. 全局修改previousPoints的声明类型为MatOfPoint2f
2. 光流计算前增加点集非空校验
3. 光流计算完成后，仅把跟踪有效的点赋值给previousPoints，避免无效点累积

修改后的关键代码 #

// 全局变量声明修改为
private MatOfPoint2f previousPoints = new MatOfPoint2f();
private Mat matPrevGray = new Mat();

// ... 中间Haar检测逻辑不变 ...
Rect[] carsArray = cars.toArray();
ArrayList<Point> CurrentCars = new ArrayList<>();

for (int i = 0; i < carsArray.length; i++)
{
    Imgproc.rectangle(matRGB, carsArray[i].tl(), carsArray[i].br(),
            CAR_RECT_COLOR, 3);
    int xCenter = (carsArray[i].x + carsArray[i].x + carsArray[i].width) / 2;
    int yCenter = (carsArray[i].y + carsArray[i].y + carsArray[i].height) / 2;
    Point center = new Point(xCenter, yCenter);
    CurrentCars.add(center);
    Imgproc.putText(matRGB, "[" + center.x + "," + center.y + "]",
            new Point(center.x + 20, center.y + 20),
            Core.FONT_HERSHEY_SIMPLEX, 0.7, new Scalar(255, 255, 255, 255));
}

if (previousPoints.empty()){
    if(!CurrentCars.isEmpty()){
        previousPoints.fromList(CurrentCars);
        matGray.copyTo(matPrevGray);
    }
    // 没有检测到目标直接跳过当前帧光流计算
    return;
}

// 光流计算前先校验前序点集非空
if(previousPoints.empty()){
    return;
}

MatOfPoint2f currentPoints = new MatOfPoint2f();
MatOfByte status = new MatOfByte();
MatOfFloat err = new MatOfFloat();
TermCriteria criteria = new TermCriteria(TermCriteria.COUNT + TermCriteria.EPS,10,0.03);
// 传入的前序点已经是MatOfPoint2f类型，符合接口要求
Video.calcOpticalFlowPyrLK(matPrevGray,matGray,previousPoints,currentPoints,status,err);

byte StatusArr[] = status.toArray();
Point p0Arr[] = previousPoints.toArray();
Point p1Arr[] = currentPoints.toArray();
ArrayList<Point> good_new = new ArrayList<>();
for (int i = 0; i<StatusArr.length ; i++ ) {
    if (StatusArr[i] == 1) {
        good_new.add(p1Arr[i]);
        Imgproc.line(matRGB, p1Arr[i], p0Arr[i], new Scalar(255,255,0,0),2);
        Imgproc.circle(matRGB, p1Arr[i],5, new Scalar(255,255,0,0),-1);
    }
}

// 仅用跟踪有效的点更新前序点集
if(!good_new.isEmpty()){
    previousPoints.fromList(good_new);
}else{
    // 所有点跟踪失效，重置前序点集，下一帧重新检测
    previousPoints.release();
    previousPoints = new MatOfPoint2f();
}
matGray.copyTo(matPrevGray);

内容的提问来源于stack exchange，提问作者farah