带噪点偏移的形状图像匹配失效，求有效解决方案

我有一组代表不同形状的图像，以某一张为参考图，需要从另外6张加了噪点滤镜且未完全居中的图像中找出最相似的目标图。参考相关代码实现后，匹配结果错误——实际正确匹配是图像4，但算法输出图像6相似度最高。

原实现代码 #

from skimage.metrics import structural_similarity
import cv2
from PIL import Image

#适用于不同尺寸的图像
def orb_sim(img1, img2):
  orb = cv2.ORB_create()
  # 检测关键点和描述符
  kp_a, desc_a = orb.detectAndCompute(img1, None)
  kp_b, desc_b = orb.detectAndCompute(img2, None)
  # 定义暴力匹配器对象
  bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True) 
  #执行匹配
  matches = bf.match(desc_a, desc_b)
  #筛选距离<50的相似区域，距离范围0-100
  similar_regions = [i for i in matches if i.distance < 50]  
  if len(matches) == 0:
    return 0
  return len(similar_regions) / len(matches)

#要求图像尺寸相同
def structural_sim(img1, img2):
  sim, diff = structural_similarity(img1, img2, full=True)
  return sim

im_ref = cv2.imread('captcha_clean/umbrella.png', 0)
for i in range(1,7):
  path = "Tmp_Image/split_"+str(i)+".png"
  im = Image.open(path)
  path = "Tmp_Image/newsplit_"+str(i)+".png"
  new_im = im.resize((32,32))
  new_im.save(path)
  im_test = cv2.imread(path, 0)
  ssim = structural_sim(im_ref, im_test) #1.0表示完全相同，值越小相似度越低
  print("For image "+str(i)+" similarity using SSIM is: ", ssim)
  orb_similarity = orb_sim(im_ref, im_test)  #1.0表示完全相同，值越小相似度越低
  print("For image "+str(i)+" similarity using ORB is: ", orb_similarity)

错误输出结果 #

For image 1 similarity using SSIM is:  -0.04562843656475159
For image 1 similarity using ORB is:  0
For image 2 similarity using SSIM is:  0.04770572948037391
For image 2 similarity using ORB is:  0
For image 3 similarity using SSIM is:  0.10395830102945436
For image 3 similarity using ORB is:  0
For image 4 similarity using SSIM is:  0.08297170823406234
For image 4 similarity using ORB is:  0
For image 5 similarity using SSIM is:  0.07766704880294842
For image 5 similarity using ORB is:  0
For image 6 similarity using SSIM is:  0.12072132618711812
For image 6 similarity using ORB is:  0

解决方案 #

1. 预处理优化：对齐与降噪 #

• 图像对齐：直接缩放会因目标未居中导致形状错位，先做轮廓检测+中心对齐：
  1. 对测试图二值化，提取最大轮廓
  2. 计算轮廓外接矩形，将轮廓中心移至图像中心，生成对齐后的图像
• 降噪处理：用cv2.GaussianBlur或cv2.medianBlur去除噪点，减少算法干扰

2. 改进ORB匹配逻辑 #

原ORB返回0是因为噪点干扰关键点匹配，调整参数并改用KNN匹配：

import numpy as np

def orb_sim(img1, img2):
    # 调整ORB参数，增加关键点数量、降低边缘阈值
    orb = cv2.ORB_create(edgeThreshold=15, nfeatures=1000)
    kp_a, desc_a = orb.detectAndCompute(img1, None)
    kp_b, desc_b = orb.detectAndCompute(img2, None)
    
    # 处理无描述符的情况
    if desc_a is None or desc_b is None or len(desc_a) < 2 or len(desc_b) < 2:
        return 0
    
    # 改用KNN匹配+比率测试筛选有效匹配
    bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=False)
    matches = bf.knnMatch(desc_a, desc_b, k=2)
    
    good_matches = []
    for m, n in matches:
        if m.distance < 0.75 * n.distance:
            good_matches.append(m)
    
    return len(good_matches) / len(matches) if matches else 0

3. 改用形状鲁棒的匹配算法 #

Hu矩匹配（对缩放/平移/旋转不变）

def hu_moments_sim(img1, img2):
    # 自适应二值化，突出形状轮廓
    _, img1_bin = cv2.threshold(img1, 127, 255, cv2.THRESH_BINARY_INV if np.mean(img1) > 127 else cv2.THRESH_BINARY)
    _, img2_bin = cv2.threshold(img2, 127, 255, cv2.THRESH_BINARY_INV if np.mean(img2) > 127 else cv2.THRESH_BINARY)
    
    # 计算Hu矩并做对数变换
    moments1 = cv2.moments(img1_bin)
    hu_moments1 = cv2.HuMoments(moments1)
    hu_moments1 = -np.sign(hu_moments1) * np.log10(np.abs(hu_moments1))
    
    moments2 = cv2.moments(img2_bin)
    hu_moments2 = cv2.HuMoments(moments2)
    hu_moments2 = -np.sign(hu_moments2) * np.log10(np.abs(hu_moments2))
    
    # 计算欧氏距离，转为相似度（值越大越相似）
    distance = np.linalg.norm(hu_moments1 - hu_moments2)
    return 1 / (1 + distance)

归一化模板匹配

def template_match_sim(img_ref, img_test):
    # 确保测试图尺寸足够，否则缩放参考图适配
    if img_test.shape[0] < img_ref.shape[0] or img_test.shape[1] < img_ref.shape[1]:
        img_ref = cv2.resize(img_ref, (img_test.shape[1], img_test.shape[0]))
    
    # 使用归一化相关系数匹配，值越接近1越相似
    result = cv2.matchTemplate(img_test, img_ref, cv2.TM_CCOEFF_NORMED)
    return np.max(result)

4. 多算法综合评分 #

将SSIM、改进ORB、Hu矩的相似度加权计算，提升准确率：

def combined_similarity(img_ref, img_test):
    ssim = structural_sim(img_ref, img_test)
    orb = orb_sim(img_ref, img_test)
    hu = hu_moments_sim(img_ref, img_test)
    
    # 加权分配，可根据实际情况调整权重
    return 0.2 * ssim + 0.3 * orb + 0.5 * hu

内容的提问来源于stack exchange，提问作者TimeTraveler87