Mediapipe握拳手势识别无法与自定义GUI交互问题求助

一、实现握拳状态判断 #

利用Mediapipe输出的手部关键点，通过对比指尖与指根的相对坐标判断握拳：

def is_fist(hand_landmarks, img_width):
    # 食指到小指的指尖/指根索引配对
    finger_pairs = [(8,5), (12,9), (16,13), (20,17)]
    # 检查四指是否弯曲（Mediapipe y轴从上到下递增，指尖在指根下方则为弯曲）
    for tip_idx, root_idx in finger_pairs:
        tip_y = hand_landmarks.landmark[tip_idx].y
        root_y = hand_landmarks.landmark[root_idx].y
        if tip_y < root_y:
            return False
    # 补充拇指判断（右手场景：拇指指尖在指根左侧，左手可反转逻辑）
    thumb_tip = hand_landmarks.landmark[4]
    thumb_root = hand_landmarks.landmark[2]
    if thumb_tip.x > thumb_root.x and (img_width - thumb_tip.x * img_width) > 50:
        return False
    return True

二、实现画圈轨迹检测 #

通过追踪握拳状态下的关键点移动轨迹，判断是否形成闭合圈：

import math

# 存储轨迹的列表
trajectory = []

def update_trajectory(hand_landmarks, img_width, img_height, is_fist_state):
    global trajectory
    if is_fist_state:
        # 取手腕关键点作为轨迹参考点
        wrist = hand_landmarks.landmark[0]
        x = int(wrist.x * img_width)
        y = int(wrist.y * img_height)
        trajectory.append((x, y))
    else:
        # 握拳松开时校验轨迹是否为圈
        if is_circle(trajectory):
            trigger_menu_navigation()
        # 清空轨迹，准备下一次检测
        trajectory = []

def is_circle(trajectory):
    # 轨迹点数量过少直接排除
    if len(trajectory) < 25:
        return False
    # 计算轨迹总长度
    total_dist = 0
    for i in range(1, len(trajectory)):
        x1, y1 = trajectory[i-1]
        x2, y2 = trajectory[i]
        total_dist += math.hypot(x2 - x1, y2 - y1)
    # 计算起点与终点的距离，判断轨迹是否闭合
    start_x, start_y = trajectory[0]
    end_x, end_y = trajectory[-1]
    end_dist = math.hypot(end_x - start_x, end_y - start_y)
    # 阈值可根据实际场景调整
    return total_dist > 150 and end_dist < 40

三、关联GUI菜单交互 #

在主循环中将手势识别结果与菜单逻辑绑定：

# 假设你的自定义菜单类为CustomMenu
menu = CustomMenu()

def trigger_menu_navigation():
    # 替换为你的菜单导航逻辑，比如切换选项、进入子菜单
    if menu.current_state == "main":
        menu.enter_submenu()
    else:
        menu.back_to_main()

# 主循环中整合手势识别与菜单绘制
while cap.isOpened():
    success, img = cap.read()
    if not success:
        continue
    img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    results = hands.process(img_rgb)
    img_height, img_width = img.shape[:2]
    is_fist_state = False
    
    if results.multi_hand_landmarks:
        for hand_landmarks in results.multi_hand_landmarks:
            # 保留原有关键点绘制代码
            mp_drawing.draw_landmarks(img, hand_landmarks, mp_hands.HAND_CONNECTIONS)
            # 判断当前是否握拳
            is_fist_state = is_fist(hand_landmarks, img_width)
    
    # 更新轨迹并触发菜单逻辑
    if results.multi_hand_landmarks:
        update_trajectory(results.multi_hand_landmarks[0], img_width, img_height, is_fist_state)
    
    # 保留原有菜单绘制代码
    draw_menu(img, menu)
    
    cv2.imshow('Hand Gesture Menu', img)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

关键注意事项 #

• 坐标转换：Mediapipe输出的是0-1的归一化坐标，必须乘以图像宽高转换为像素坐标后才能用于轨迹计算
• 阈值调整：is_circle中的轨迹总长度、闭合距离阈值，is_fist中的拇指判断阈值，需根据拍摄环境手动调试
• 防抖处理：可添加连续帧判断逻辑（比如连续3帧检测到握拳才开始记录轨迹），减少误触发
• 左手适配：若需支持左手，修改is_fist中的拇指判断逻辑，反转x坐标的比较方向

内容的提问来源于stack exchange，提问作者Dolfie