如何使用OpenCV在不检测色彩的情况下在图像中查找目标图像？

我正在尝试在一张图像（待查找图）中定位另一张图像（目标图）的坐标，目前只有当两者颜色完全相同时才能正常匹配，但我需要实现不受色彩差异影响的定位功能。

我试过用灰度化+模板匹配的方案，代码如下：

import cv from "@u4/opencv4nodejs";

const MIN_ACCURACY = 0.8;

export async function getImageCoordinates(source: Buffer, template: Buffer) {
  const haystack = await cv.imdecodeAsync(source, cv.IMREAD_GRAYSCALE);
  const needle = await cv.imdecodeAsync(template, cv.IMREAD_GRAYSCALE);

  const matched = await haystack.matchTemplateAsync(
    needle,
    cv.TM_CCOEFF_NORMED
  );

  const minMax = await matched.minMaxLocAsync();
  const { x, y } = minMax.maxLoc;
  const accuracy = minMax.maxVal;

  if (accuracy > MIN_ACCURACY) {
    return { x, y };
  }

  return undefined;
}

但当两张图颜色不同时，灰度色调会存在明显差异，导致匹配精度不够，无法正常定位。

示例场景：

• 待查找图像：包含目标物体但颜色与模板不同的大图
• 目标图像：作为模板的小图

可行改进方案 #

1. 基于边缘检测的模板匹配 #

先对两张图提取边缘信息，再做模板匹配，忽略颜色差异，只匹配物体的轮廓特征：

import cv from "@u4/opencv4nodejs";

const MIN_ACCURACY = 0.7;
const CANNY_THRESHOLD1 = 50;
const CANNY_THRESHOLD2 = 150;

export async function getImageCoordinates(source: Buffer, template: Buffer) {
  // 解码为彩色图
  const haystackColor = await cv.imdecodeAsync(source);
  const needleColor = await cv.imdecodeAsync(template);

  // 转为灰度图
  const haystackGray = haystackColor.cvtColor(cv.COLOR_BGR2GRAY);
  const needleGray = needleColor.cvtColor(cv.COLOR_BGR2GRAY);

  // 进行Canny边缘检测
  const haystackEdges = haystackGray.canny(CANNY_THRESHOLD1, CANNY_THRESHOLD2);
  const needleEdges = needleGray.canny(CANNY_THRESHOLD1, CANNY_THRESHOLD2);

  // 边缘图模板匹配
  const matched = await haystackEdges.matchTemplateAsync(
    needleEdges,
    cv.TM_CCOEFF_NORMED
  );

  const minMax = await matched.minMaxLocAsync();
  const { x, y } = minMax.maxLoc;
  const accuracy = minMax.maxVal;

  if (accuracy > MIN_ACCURACY) {
    return { x, y };
  }

  return undefined;
}

注：可根据实际图像调整Canny阈值和匹配精度阈值。

2. 基于特征点的匹配（更鲁棒） #

如果边缘检测仍不够稳定，可使用ORB/SIFT等特征点匹配算法，提取图像局部特征点进行匹配，不受整体颜色、缩放、旋转影响：

import cv from "@u4/opencv4nodejs";

export async function getImageCoordinates(source: Buffer, template: Buffer) {
  // 解码图像
  const haystack = await cv.imdecodeAsync(source);
  const needle = await cv.imdecodeAsync(template);

  // 初始化ORB特征检测器
  const orb = cv.ORB.create(500); // 特征点数量可调整

  // 检测特征点并计算描述符
  const [kpHaystack, desHaystack] = orb.detectAndComputeAsync(haystack, null);
  const [kpNeedle, desNeedle] = orb.detectAndComputeAsync(needle, null);

  // 使用暴力匹配器
  const matcher = cv.BFMatcher.create(cv.NORM_HAMMING, true);
  const matches = await matcher.matchAsync(desNeedle, desHaystack);

  // 筛选优质匹配
  const goodMatches = matches.filter(match => match.distance < 50); // 距离阈值可调

  if (goodMatches.length > 10) { // 足够多匹配点才判定成功
    // 获取匹配点坐标
    const ptsNeedle = goodMatches.map(m => kpNeedle[m.queryIdx].pt);
    const ptsHaystack = goodMatches.map(m => kpHaystack[m.trainIdx].pt);

    // 计算单应性矩阵
    const homography = cv.findHomography(
      cv.matFromArray(ptsNeedle, cv.CV_32F),
      cv.matFromArray(ptsHaystack, cv.CV_32F),
      cv.RANSAC,
      5.0
    );

    // 获取模板在待查找图中的左上角坐标
    const templateCorners = [
      new cv.Point(0, 0),
      new cv.Point(needle.cols, 0),
      new cv.Point(needle.cols, needle.rows),
      new cv.Point(0, needle.rows)
    ];
    const transformedCorners = cv.perspectiveTransform(
      cv.matFromArray(templateCorners, cv.CV_32F),
      homography
    );

    // 返回左上角坐标
    return { 
      x: transformedCorners.getDataAsArray()[0][0], 
      y: transformedCorners.getDataAsArray()[0][1] 
    };
  }

  return undefined;
}

内容的提问来源于stack exchange，提问作者user26121356