如何基于dlib面部关键点在OpenCV中实现眼周平滑闭合折线蒙版

Hey there! I’ve dealt with exactly this problem before—getting those janky polyline edges to look smooth around the under-eye area can be tricky, but there are a couple of solid approaches to fix it. Let’s break down the best methods using your existing dlib keypoints and OpenCV.

Method 1: B-Spline Interpolation (Best for Precise Smooth Curves) #

This is my go-to because it generates a natural, smooth curve that follows your keypoints closely without looking rigid. We’ll use scipy’s spline functions to create a dense set of smooth points from your original keypoints.

First, pick the right keypoints from dlib’s 68-point model. For the under-eye ROI, here’s a reliable set of indices (adjust based on your exact desired shape):

• Left cheek to left eye: [3, 37, 38, 39, 40, 41]
• Nose bridge to connect both sides: [48, 60, 64]
• Right eye to right cheek: [46, 45, 44, 43, 13] (reversed to keep the curve continuous)

Here’s the code to turn those points into a smooth mask:

import numpy as np
import cv2
from scipy.interpolate import make_interp_spline

# Assume you already have dlib's shape predictor output stored in 'shape'
# Step 1: Define your ROI keypoint indices (adjust as needed)
roi_indices = [3, 37, 38, 39, 40, 41, 48, 60, 64, 46, 45, 44, 43, 13]

# Step 2: Extract coordinates from dlib shape
original_points = []
for idx in roi_indices:
    x = shape.part(idx).x
    y = shape.part(idx).y
    original_points.append([x, y])
original_points = np.array(original_points, dtype=np.float32)

# Step 3: Generate smooth spline points
# Create parameter t for interpolation
t = np.linspace(0, 1, len(original_points))
# Generate dense points (100 balances smoothness and speed)
t_smooth = np.linspace(0, 1, 100)

# Fit cubic splines to x and y coordinates
spline_x = make_interp_spline(t, original_points[:, 0], k=3)
spline_y = make_interp_spline(t, original_points[:, 1], k=3)

x_smooth = spline_x(t_smooth)
y_smooth = spline_y(t_smooth)

# Convert to OpenCV's required shape (n, 1, 2) and int32 type
smooth_points = np.column_stack((x_smooth, y_smooth)).astype(np.int32)
smooth_points = smooth_points.reshape((-1, 1, 2))

# Step 4: Draw the smooth mask
mask = np.zeros_like(your_input_image)
cv2.fillPoly(mask, [smooth_points], (255, 255, 255))

# Optional: Add slight Gaussian blur to soften edges further
mask = cv2.GaussianBlur(mask, (5, 5), 0)
_, mask = cv2.threshold(mask, 127, 255, cv2.THRESH_BINARY)

Method 2: Post-Processing Mask for Smoothness (Quick Fix) #

If you don’t want to add scipy as a dependency, you can start with your existing polyline mask and smooth it using OpenCV’s image processing tools:

# Assume you already have your initial polyline mask stored in 'rough_mask'
# Step 1: Apply Gaussian blur to soften edges
blurred_mask = cv2.GaussianBlur(rough_mask, (11, 11), 0)
# Step 2: Threshold to get back a binary mask
_, smooth_mask = cv2.threshold(blurred_mask, 127, 255, cv2.THRESH_BINARY)
# Optional: Use morphological closing to fill small gaps
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
smooth_mask = cv2.morphologyEx(smooth_mask, cv2.MORPH_CLOSE, kernel)

This method is simpler but less precise—it might slightly distort your ROI shape, so use it for quick wins.

Key Tips: #

• Keypoint Order Matters: Ensure your indices are ordered in a continuous clockwise/counter-clockwise loop around the ROI. Out-of-order points will twist the spline.
• Adjust Spline Degree: k=3 (cubic spline) gives the most natural facial curve. Use k=2 for a tighter fit, or k=4 for even smoother (but looser) curves.
• Point Density: Increasing smooth points (e.g., from 100 to 200) boosts smoothness, but 100 is usually enough.

That should give you the smooth under-eye mask you’re aiming for. Feel free to tweak the keypoint indices to match your exact use case!

内容的提问来源于stack exchange，提问作者Mobeen