如何在Google Colab中运行代码实现带关键点的实时人体捕捉？

你的代码存在几个适配Colab环境的问题：缺少核心检测和绘制函数、Colab不支持cv2.imshow()、无法直接调用本地摄像头，以下是完整的修复和实现方案：

1. 安装依赖（确保环境配置正确） #

首先执行以下命令安装必要库：

!pip install opencv-python mediapipe

2. 补充缺失的核心函数 #

原代码中缺少mediapipe_detection和draw_styled_landmarks函数，这两个是MediaPipe姿态捕捉的核心逻辑：

import cv2
import mediapipe as mp
from google.colab.patches import cv2_imshow
import numpy as np

# MediaPipe检测函数：处理图像颜色空间并运行模型
def mediapipe_detection(image, model):
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)  # 转换为MediaPipe支持的RGB格式
    image.flags.writeable = False                   # 禁用写入提升检测性能
    results = model.process(image)                  # 运行姿态检测
    image.flags.writeable = True                    # 重新启用写入
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)  # 转回OpenCV的BGR格式
    return image, results

# 带样式的关键点绘制函数：绘制面部、身体、手部的关键点及连接
mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles

def draw_styled_landmarks(image, results):
    # 绘制面部轮廓关键点
    mp_drawing.draw_landmarks(
        image,
        results.face_landmarks,
        mp_holistic.FACEMESH_CONTOURS,
        landmark_drawing_spec=None,
        connection_drawing_spec=mp_drawing_styles.get_default_face_mesh_contours_style())
    # 绘制身体姿态关键点
    mp_drawing.draw_landmarks(
        image,
        results.pose_landmarks,
        mp_holistic.POSE_CONNECTIONS,
        landmark_drawing_spec=mp_drawing_styles.get_default_pose_landmarks_style())
    # 绘制左右手关键点
    for hand_landmarks in [results.left_hand_landmarks, results.right_hand_landmarks]:
        mp_drawing.draw_landmarks(
            image,
            hand_landmarks,
            mp_holistic.HAND_CONNECTIONS,
            landmark_drawing_spec=mp_drawing_styles.get_default_hand_landmarks_style())

3. 适配Colab的实时摄像头捕获逻辑 #

Colab无法直接通过cv2.VideoCapture(0)访问本地摄像头，需要通过浏览器授权获取视频流，以下是实时捕获并处理的完整代码：

mp_holistic = mp.solutions.holistic

# 导入Colab交互工具
from IPython.display import display, Javascript
from google.colab.output import eval_js
from base64 import b64decode
from google.colab import kernel

# 实时视频流处理函数
def realtime_pose_capture():
    # 生成JavaScript脚本获取浏览器摄像头流
    js = Javascript('''
        async function streamVideo() {
            const video = document.createElement('video');
            video.style.display = 'block';
            const stream = await navigator.mediaDevices.getUserMedia({video: true});
            document.body.appendChild(video);
            video.srcObject = stream;
            await video.play();

            google.colab.output.setIframeHeight(document.documentElement.scrollHeight, true);

            const canvas = document.createElement('canvas');
            canvas.width = video.videoWidth;
            canvas.height = video.videoHeight;

            while (true) {
                canvas.getContext('2d').drawImage(video, 0, 0);
                const dataUrl = canvas.toDataURL('image/jpeg', 0.8);
                const result = await google.colab.kernel.invokeFunction('notebook.process_frame', [dataUrl], {});
                if (result.data['text/plain'] === 'stop') break;
            }
            stream.getVideoTracks()[0].stop();
            video.remove();
        }
        ''')
    display(js)

# 帧处理回调函数：接收摄像头帧并运行姿态检测
def process_frame(data_url):
    # 解码base64格式的帧数据
    binary = b64decode(data_url.split(',')[1])
    image = cv2.imdecode(np.frombuffer(binary, np.uint8), cv2.IMREAD_COLOR)
    
    # 运行MediaPipe姿态检测
    with mp_holistic.Holistic(min_detection_confidence=0.5, min_tracking_confidence=0.5) as holistic:
        image, results = mediapipe_detection(image, holistic)
        draw_styled_landmarks(image, results)
    
    # 显示处理后的帧（Colab专用）
    cv2_imshow(image)
    return 'continue'

# 注册帧处理回调
kernel.register_callback('notebook.process_frame', process_frame)

# 启动实时捕获
realtime_pose_capture()

关键修改说明 #

• 替换cv2.imshow()为cv2_imshow()：Colab不支持原生OpenCV窗口显示，需使用Colab提供的专用显示函数
• 修复原代码缩进错误：原while循环内的代码未缩进，会导致语法错误
• 解决摄像头访问问题：通过JavaScript脚本获取浏览器授权的摄像头流，绕过Colab无法直接访问本地硬件的限制
• 补充核心函数：实现MediaPipe的图像预处理和带样式的关键点绘制逻辑

内容的提问来源于stack exchange，提问作者Estiaque