99%相似印刷图像非像素级差异检测技术求助

Hey there! Let's tackle your print image comparison challenge head-on. The core issue here is that pixel-level comparisons struggle with minor misalignments (even after your current alignment step) and subtle print variations, which is why you're hitting that 80-90% accuracy ceiling. Let's break this down into fixes for alignment first, then move to smarter, non-pixel-level difference detection techniques.

Fixing the Image Alignment Failure Rate #

Your current alignment code relies on contour detection, which can fail due to noise, uneven lighting, or small print artifacts. Here's how to make it more robust:

1. Preprocess to Reduce Noise #

Add Gaussian blurring and adaptive thresholding to clean up the image before contour detection:

import cv2
import numpy as np

def align_print_image(img_path):
    img = cv2.imread(img_path)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    
    # Preprocess: Gaussian blur + adaptive threshold
    blurred = cv2.GaussianBlur(gray, (5,5), 0)
    thresh = cv2.adaptiveThreshold(blurred, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 11, 2)
    
    # Find contours with hierarchy to filter out small noise
    contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    
    # Filter contours by area and aspect ratio (assume print is rectangular)
    max_area = 0
    best_contour = None
    for cnt in contours:
        area = cv2.contourArea(cnt)
        if area < 1000:  # Skip tiny noise contours
            continue
        x, y, w, h = cv2.boundingRect(cnt)
        aspect_ratio = w / h
        # Adjust aspect ratio range to match your print dimensions
        if 0.8 < aspect_ratio < 1.2 and area > max_area:
            max_area = area
            best_contour = cnt
    
    if best_contour is None:
        raise ValueError("No valid print contour found")
    
    # Get perspective transform for alignment
    rect = cv2.minAreaRect(best_contour)
    box = cv2.boxPoints(rect)
    box = np.int0(box)
    
    width = int(rect[1][0])
    height = int(rect[1][1])
    
    # Define destination points to correct rotation
    dst_pts = np.array([[0, height-1], [0, 0], [width-1, 0], [width-1, height-1]], dtype="float32")
    src_pts = box.astype("float32")
    
    M = cv2.getPerspectiveTransform(src_pts, dst_pts)
    warped = cv2.warpPerspective(img, M, (width, height))
    
    return warped

# Usage
aligned_img1 = align_print_image('./photo/image1.jpg')
aligned_img2 = align_print_image('./photo/image2.jpg')
cv2.imwrite('./aligned_image1.jpg', aligned_img1)
cv2.imwrite('./aligned_image2.jpg', aligned_img2)

2. Fallback to Feature-Based Alignment #

If contour detection still fails, use ORB feature matching for alignment—it's more robust to noise and partial occlusions:

def align_with_feature_matching(img1, img2):
    # Initialize ORB detector
    orb = cv2.ORB_create(500)
    kp1, des1 = orb.detectAndCompute(cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY), None)
    kp2, des2 = orb.detectAndCompute(cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY), None)
    
    # Match features
    bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
    matches = bf.match(des1, des2)
    matches = sorted(matches, key=lambda x: x.distance)
    
    # Extract good matches (top 10%)
    good_matches = matches[:int(len(matches)*0.1)]
    
    # Get points for homography
    src_pts = np.float32([kp1[m.queryIdx].pt for m in good_matches]).reshape(-1,1,2)
    dst_pts = np.float32([kp2[m.trainIdx].pt for m in good_matches]).reshape(-1,1,2)
    
    # Compute homography
    M, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, 5.0)
    
    # Warp image to align
    h, w = img1.shape[:2]
    aligned_img = cv2.warpPerspective(img1, M, (w, h))
    
    return aligned_img

Moving Beyond Pixel-Level Comparison #

Pixel-level checks miss the forest for the trees—they focus on individual pixel differences instead of meaningful regions. Here are three effective approaches:

1. Superpixel + Region Feature Comparison #

Split images into "superpixels" (meaningful, contiguous pixel blocks) and compare region-level features (like color histograms) instead of individual pixels. This ignores tiny pixel noise and focuses on real differences:

import cv2
import numpy as np
from skimage.segmentation import slic
from skimage.util import img_as_float
from skimage.measure import regionprops

def compare_superpixel_regions(img1, img2):
    # Convert to RGB (skimage uses RGB, OpenCV uses BGR)
    img1_rgb = cv2.cvtColor(img_as_float(img1), cv2.COLOR_BGR2RGB)
    img2_rgb = cv2.cvtColor(img_as_float(img2), cv2.COLOR_BGR2RGB)
    
    # Generate superpixels (adjust n_segments based on your image size)
    segments1 = slic(img1_rgb, n_segments=200, compactness=10)
    segments2 = slic(img2_rgb, n_segments=200, compactness=10)
    
    # Get region properties for each superpixel
    props1 = regionprops(segments1, intensity_image=img1_rgb)
    props2 = regionprops(segments2, intensity_image=img2_rgb)
    
    # Create a mask to highlight differences
    diff_mask = np.zeros_like(img1[:, :, 0], dtype=np.uint8)
    
    for prop1, prop2 in zip(props1, props2):
        # Calculate color histograms for each region
        hist1 = cv2.calcHist([img1_rgb], [0,1,2], prop1.mask.astype(np.uint8), [8,8,8], [0,1,0,1,0,1])
        hist2 = cv2.calcHist([img2_rgb], [0,1,2], prop2.mask.astype(np.uint8), [8,8,8], [0,1,0,1,0,1])
        
        # Compare histograms using Chi-Squared distance (higher = more different)
        distance = cv2.compareHist(hist1, hist2, cv2.HISTCMP_CHISQR)
        
        # Mark region as different if distance exceeds threshold (adjust based on your data)
        if distance > 10:
            diff_mask[prop1.coords[:, 0], prop1.coords[:, 1]] = 255
    
    # Clean up mask with median blur to remove small noise
    diff_mask = cv2.medianBlur(diff_mask, 5)
    return diff_mask

# Usage
diff_mask = compare_superpixel_regions(aligned_img1, aligned_img2)
cv2.imwrite('./difference_mask.png', diff_mask)

2. Feature-Based Difference Detection #

Use ORB/SIFT to find matching features between the two images. Regions with no matching features are likely the differences:

def find_feature_based_differences(img1, img2):
    orb = cv2.ORB_create(1000)
    kp1, des1 = orb.detectAndCompute(cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY), None)
    kp2, des2 = orb.detectAndCompute(cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY), None)
    
    # Match features
    bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
    matches = bf.match(des1, des2)
    
    # Create a mask of matched points
    matched_mask = np.zeros(len(kp1), dtype=np.uint8)
    for match in matches:
        matched_mask[match.queryIdx] = 1
    
    # Draw unmatched points (potential differences)
    diff_img = cv2.drawKeypoints(img1, kp1, None, color=(0,0,255), flags=cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS, mask=1 - matched_mask)
    return diff_img

3. Lightweight Deep Learning for Semantic Comparison #

For structured print images, a tiny CNN can learn to detect meaningful differences. You can fine-tune a pre-trained model like MobileNet on your labeled data (your Accepted output samples):

# Example using TensorFlow/Keras (simplified)
import tensorflow as tf
from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D
from tensorflow.keras.models import Model

# Load base model
base_model = MobileNetV2(input_shape=(224,224,3), include_top=False, weights='imagenet')

# Add custom head for difference detection
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(128, activation='relu')(x)
predictions = Dense(1, activation='sigmoid')(x)  # 1 = different, 0 = same

model = Model(inputs=base_model.input, outputs=predictions)

# Freeze base model and train only custom layers
for layer in base_model.layers:
    layer.trainable = False

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Train on your labeled dataset (aligned image pairs + labels: same/different)
# model.fit(train_data, train_labels, epochs=10, validation_data=(val_data, val_labels))

Additional Tips for Better Accuracy #

• Normalize Lighting: Use CLAHE (Contrast Limited Adaptive Histogram Equalization) to reduce lighting variations before processing:

  clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
  normalized_gray = clahe.apply(gray)
  

• Focus on Relevant Regions: Use text detection (like EAST detector) to isolate text areas, then compare those regions separately—this ignores irrelevant background noise.
• Tune Thresholds: Use your Accepted output samples to adjust thresholds (like histogram distance, feature matching distance) for optimal accuracy.

内容的提问来源于stack exchange，提问作者mithil