如何基于人脸关键点检测脸型？TensorFlow.js项目实现指导

核心思路 #

通过MediaPipe FaceMesh输出的468个关键点，提取面部关键维度（宽度、高度、下颌线形态）的比例特征，以此匹配常见脸型（圆形、椭圆形、方形、心形等）的特征阈值。

关键步骤与代码实现 #

1. 定义关键关键点索引

MediaPipe FaceMesh的关键点有固定索引，我们需要提取以下核心点位：

• 头顶：10
• 下巴尖：152
• 左颧骨最宽处：234
• 右颧骨最宽处：454
• 左下颌角：212
• 右下颌角：432

2. 计算面部比例并分类

编写工具函数，输入人脸关键点数据，输出对应的脸型：

// 计算两点之间的欧氏距离
const calculateDistance = (point1, point2) => {
  const dx = point1.x - point2.x;
  const dy = point1.y - point2.y;
  return Math.sqrt(dx * dx + dy * dy);
};

// 根据关键点判断脸型
const classifyFaceShape = (landmarks) => {
  // 提取核心点位
  const forehead = landmarks[10];
  const chin = landmarks[152];
  const leftCheek = landmarks[234];
  const rightCheek = landmarks[454];
  const leftJaw = landmarks[212];
  const rightJaw = landmarks[432];

  // 计算核心维度比例
  const faceWidth = calculateDistance(leftCheek, rightCheek);
  const faceHeight = calculateDistance(forehead, chin);
  const jawWidth = calculateDistance(leftJaw, rightJaw);
  const widthHeightRatio = faceWidth / faceHeight;
  const jawCheekRatio = jawWidth / faceWidth;

  // 基于比例阈值分类
  if (widthHeightRatio > 0.85) {
    // 宽高比接近1，区分方形/圆形
    return jawCheekRatio > 0.9 ? "方形" : "圆形";
  } else {
    // 宽高比偏小，区分心形/椭圆形
    return jawCheekRatio < 0.85 ? "心形" : "椭圆形";
  }
};

3. 整合到检测逻辑中

优化原有检测循环（加入requestAnimationFrame避免性能问题），并在检测后调用脸型分类函数：

import * as faceLandmarksDetection from "@tensorflow-models/face-landmarks-detection";

export const runDetector = async (video, canvas) => {
  const model = faceLandmarksDetection.SupportedModels.MediaPipeFaceMesh;
  const detectorConfig = {
    runtime: "tfjs",
    // 可选：使用高精度模型，modelType: 'full'（默认lite）
  };
  const detector = await faceLandmarksDetection.createDetector(model, detectorConfig);

  const detect = async () => {
    const estimationConfig = { flipHorizontal: false };
    const faces = await detector.estimateFaces(video, estimationConfig);

    if (faces.length > 0) {
      // 处理第一个检测到的人脸
      const face = faces[0];
      const faceShape = classifyFaceShape(face.keypoints);
      console.log("当前脸型：", faceShape);

      // 可选：在画布上可视化结果
      const ctx = canvas.getContext('2d');
      ctx.clearRect(0, 0, canvas.width, canvas.height);
      // 绘制关键点
      face.keypoints.forEach(point => {
        ctx.beginPath();
        ctx.arc(point.x, point.y, 2, 0, 2 * Math.PI);
        ctx.fillStyle = 'blue';
        ctx.fill();
      });
      // 绘制脸型标签
      ctx.font = '20px Arial';
      ctx.fillStyle = 'red';
      ctx.fillText(`脸型：${faceShape}`, 10, 30);
    }

    // 循环检测
    requestAnimationFrame(detect);
  };

  detect();
};

// 复用工具函数
const calculateDistance = (point1, point2) => {
  const dx = point1.x - point2.x;
  const dy = point1.y - point2.y;
  return Math.sqrt(dx * dx + dy * dy);
};

const classifyFaceShape = (landmarks) => {
  const forehead = landmarks[10];
  const chin = landmarks[152];
  const leftCheek = landmarks[234];
  const rightCheek = landmarks[454];
  const leftJaw = landmarks[212];
  const rightJaw = landmarks[432];

  const faceWidth = calculateDistance(leftCheek, rightCheek);
  const faceHeight = calculateDistance(forehead, chin);
  const jawWidth = calculateDistance(leftJaw, rightJaw);
  const widthHeightRatio = faceWidth / faceHeight;
  const jawCheekRatio = jawWidth / faceWidth;

  if (widthHeightRatio > 0.85) {
    return jawCheekRatio > 0.9 ? "方形" : "圆形";
  } else {
    return jawCheekRatio < 0.85 ? "心形" : "椭圆形";
  }
};

注意事项 #

• 比例阈值可根据实际测试调整，不同人种、年龄段的脸型特征存在差异
• 若需更精准分类，可增加更多特征维度（如颧骨到下巴的距离、眉骨宽度等），或引入轻量机器学习模型做分类
• 确保视频流尺寸与画布匹配，避免关键点坐标偏移

内容的提问来源于stack exchange，提问作者0x307