求助：如何移除太阳镜反光并更换镜片颜色？现有OpenCV+KMeans实现代码失效问题排查

Hey there! Let's work through your sunglasses lens color replacement project—sounds like a cool task, and I can spot a few issues in your current code that are probably causing it to fail. Let's break this down step by step.

First, Fix the Color Detection Code #

Your K-means color clustering code is missing a critical function (RGB2HEX)—that's definitely throwing an error right off the bat. Also, hardcoding crop coordinates only works for one specific image, and you're not accounting for glare (which will skew your main color detection). Here's a revised version with fixes:

import cv2
from google.colab.patches import cv2_imshow
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt
import numpy as np
from collections import Counter
from skimage.color import rgb2lab, deltaE_cie76
import os

# Add the missing RGB to HEX conversion function
def RGB2HEX(color):
    return "#{:02x}{:02x}{:02x}".format(int(color[0]), int(color[1]), int(color[2]))

img = cv2.imread('originalimage.jpg')
# Note: Hardcoded crops are fragile—you might want to use face detection to auto-find lenses later
cropped_lens2 = img[556:2045, 2000:3177]
image = cv2.cvtColor(cropped_lens2, cv2.COLOR_BGR2RGB)
modified_image = cv2.resize(image, (600, 400), interpolation = cv2.INTER_AREA)
cv2_imshow(modified_image)

# Preprocess to reduce glare impact: mask out bright highlights
gray = cv2.cvtColor(modified_image, cv2.COLOR_RGB2GRAY)
glare_mask = cv2.threshold(gray, 220, 255, cv2.THRESH_BINARY)[1]
# Inpaint to remove glare from the lens region
glare_removed = cv2.inpaint(modified_image, glare_mask, 3, cv2.INPAINT_TELEA)

# Reshape for K-means
modified_image_reshaped = glare_removed.reshape(glare_removed.shape[0]*glare_removed.shape[1], 3)
number_of_colors = 2
clf = KMeans(n_clusters = number_of_colors)
labels = clf.fit_predict(modified_image_reshaped)

counts = Counter(labels)
# Sort colors by count to get the dominant lens color first
ordered_colors = [clf.cluster_centers_[i] for i in sorted(counts.keys(), key=lambda x: counts[x], reverse=True)]
hex_colors = [RGB2HEX(color) for color in ordered_colors]
rgb_colors = [color for color in ordered_colors]

plt.figure(figsize = (8, 6))
plt.pie(counts.values(), labels = hex_colors, colors = hex_colors)
plt.title("Dominant Lens Colors")
plt.show()

# Store the dominant lens color (first in the sorted list)
dominant_color = rgb_colors[0]
print(f"Detected dominant lens color (RGB): {dominant_color}")

Next, Improve the Color Replacement Logic #

Your current PIL code uses a fixed red-channel range (0-80) to replace colors, which won't work for different lens shades. Instead, let's use CIE76 color distance (from skimage) to match pixels close to your detected dominant color, and use a lens region mask to avoid modifying other parts of the image.

Here's the optimized replacement code, integrated with the detection step:

from PIL import Image
import numpy as np

# Load the full original image
img = Image.open("originalimage.jpg").convert("RGB")
img_np = np.array(img)

# Convert dominant color to LAB for better color similarity checks
dominant_color_lab = rgb2lab(np.uint8(np.asarray([[dominant_color]])))[0][0]

# Define a color similarity threshold (adjust based on your needs)
color_threshold = 30

# Create mask for the lens region (matches your crop coordinates)
mask = np.zeros(img_np.shape[:2], dtype=bool)
mask[556:2045, 2000:3177] = True

# Convert entire image to LAB
img_lab = rgb2lab(img_np)
# Calculate distance from dominant color for each pixel
distances = deltaE_cie76(dominant_color_lab, img_lab)
# Create a mask for pixels similar to dominant color within threshold
color_match_mask = (distances < color_threshold) & mask

# Replace matching pixels with your desired color (e.g., gold: (255, 204, 100))
img_np[color_match_mask] = (255, 204, 100)

# Convert back to PIL Image
new_img = Image.fromarray(img_np)
new_img.show()
new_img.save("modified_sunglasses.jpg")

Key Fixes & Improvements #

• Added the missing RGB2HEX function to resolve the initial code error
• Added glare removal using inpainting to ensure dominant color detection isn't skewed by bright reflections
• Replaced fixed color ranges with color similarity checks (CIE76) which are far more accurate for matching real-world colors
• Used a lens region mask to ensure we only modify the lens area, not the rest of the image
• Vectorized the color replacement step for much faster processing (no slow loops over every pixel)

Next Steps for Better Accuracy #

• Instead of hardcoding crop coordinates, use a face detection model (like OpenCV's Haar cascades or MTCNN) to automatically locate the sunglasses lenses
• Adjust the color_threshold value based on your specific images—lower values mean stricter color matching, higher values mean broader replacement
• For more precise glare removal, you can try using a polarizing filter in your input images, or use advanced inpainting techniques like DeepFill

内容的提问来源于stack exchange，提问作者CHETHAN S