如何在Python中使用MediaPipe在检测到的关键点上叠加自定义形状?
如何在Python中使用MediaPipe在检测到的关键点上叠加自定义形状?
嘿,我正好折腾过类似的需求,这就给你一步步讲清楚怎么实现,完全贴合你的诉求!
先回答你的两个核心问题:
1. 怎么叠加自定义形状或图像到特定关键点?
MediaPipe给出的关键点坐标是归一化的(0-1范围),所以第一步要把它们转换成OpenCV帧的像素坐标,公式很简单:
x_pixel = int(landmark.x * 帧宽度) y_pixel = int(landmark.y * 帧高度)
拿到像素坐标后,就可以用OpenCV的各种绘图函数(比如cv2.circle()、cv2.rectangle())画自定义形状,要是想叠加图像(比如星星图标),只要处理好透明PNG的alpha通道就能自然融合。
2. 能不能完全跳过mp_drawing.draw_landmarks(),手动绘制所有内容?
当然可以!而且这样你能100%掌控每一个细节——只要遍历所有关键点画形状,再遍历mp_hands.HAND_CONNECTIONS里的连接对,用cv2.line()画连线就行,完全替代默认的绘制逻辑。
给你写了一份完整的示例代码,包含所有你要的功能
我把自定义形状、手动绘制连接、叠加透明图像都整合进去了,你可以直接跑:
import cv2 import mediapipe as mp import numpy as np mp_hands = mp.solutions.hands # 可选:加载自定义透明图像(比如星星图标,记得用PNG格式) # 要是没有这个图,注释掉相关代码就行 star_img = cv2.imread('star.png', cv2.IMREAD_UNCHANGED) star_size = 50 star_img = cv2.resize(star_img, (star_size, star_size)) cap = cv2.VideoCapture(0) # 先获取摄像头帧的宽高,后面转坐标要用 frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) with mp_hands.Hands() as hands: while cap.isOpened(): ret, frame = cap.read() if not ret: break # MediaPipe要求输入RGB帧,所以先转换 frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) results = hands.process(frame_rgb) # 转回BGR给OpenCV显示 frame_bgr = cv2.cvtColor(frame_rgb, cv2.COLOR_RGB2BGR) if results.multi_hand_landmarks: for hand_landmarks in results.multi_hand_landmarks: # -------------------------- # 手动绘制手部连接(替代默认的HAND_CONNECTIONS) # -------------------------- for connection in mp_hands.HAND_CONNECTIONS: # 拿到连接的两个关键点索引 start_idx, end_idx = connection # 转成像素坐标 start_point = ( int(hand_landmarks.landmark[start_idx].x * frame_width), int(hand_landmarks.landmark[start_idx].y * frame_height) ) end_point = ( int(hand_landmarks.landmark[end_idx].x * frame_width), int(hand_landmarks.landmark[end_idx].y * frame_height) ) # 用蓝色粗线画连接 cv2.line(frame_bgr, start_point, end_point, (255, 0, 0), 3) # -------------------------- # 给特定关键点加自定义形状 # -------------------------- # 1. 手腕位置画大红色实心圆 wrist = hand_landmarks.landmark[mp_hands.HandLandmark.WRIST] wrist_x = int(wrist.x * frame_width) wrist_y = int(wrist.y * frame_height) cv2.circle(frame_bgr, (wrist_x, wrist_y), 18, (0, 0, 255), -1) # 2. 食指指尖画黄色实心正方形 index_tip = hand_landmarks.landmark[mp_hands.HandLandmark.INDEX_FINGER_TIP] index_x = int(index_tip.x * frame_width) index_y = int(index_tip.y * frame_height) square_size = 35 cv2.rectangle( frame_bgr, (index_x - square_size//2, index_y - square_size//2), (index_x + square_size//2, index_y + square_size//2), (0, 255, 255), -1 ) # 3. 中指指尖叠加自定义星星图像(带透明通道) middle_tip = hand_landmarks.landmark[mp_hands.HandLandmark.MIDDLE_FINGER_TIP] middle_x = int(middle_tip.x * frame_width) middle_y = int(middle_tip.y * frame_height) # 计算图像粘贴的位置(避免超出帧边界) x1 = max(0, middle_x - star_size//2) y1 = max(0, middle_y - star_size//2) x2 = min(frame_width, x1 + star_size) y2 = min(frame_height, y1 + star_size) # 处理透明通道,实现自然叠加 if star_img is not None: alpha_star = star_img[:, :, 3] / 255.0 alpha_frame = 1.0 - alpha_star # 只在图像区域内做颜色混合 for c in range(3): frame_bgr[y1:y2, x1:x2, c] = ( alpha_star[:y2-y1, :x2-x1] * star_img[:y2-y1, :x2-x1, c] + alpha_frame[:y2-y1, :x2-x1] * frame_bgr[y1:y2, x1:x2, c] ) cv2.imshow("Custom Hand Visualization", frame_bgr) if cv2.waitKey(10) & 0xFF == ord('q'): break cap.release() cv2.destroyAllWindows()
几个你要注意的细节:
- 要是不用自定义图像,直接把相关代码注释掉就行,不影响其他功能。
- 所有关键点的索引都可以通过
mp_hands.HandLandmark枚举来调用,比如拇指指尖是mp_hands.HandLandmark.THUMB_TIP,很直观。 - 叠加透明图像时,一定要用
cv2.IMREAD_UNCHANGED加载,这样才能保留alpha通道,不然会出现白色背景。
备注:内容来源于stack exchange,提问作者anfas2




