Flutter人脸识别后裁剪人脸图片显示异常问题求助

核心问题分析 #

人脸检测正常说明InputImage的坐标系是正确的，但图像显示错乱的根源在于：

• NV21转imgLib.Image时未正确处理像素行跨度（bytesPerRow）或YUV转RGB的格式转换
• 旋转图像后未同步转换人脸矩形的坐标，导致裁剪区域与实际人脸位置不匹配

分步解决方案 #

1. 替换NV21转imgLib.Image的转换函数 #

原始转换可能忽略了CameraImage的bytesPerRow（部分设备的行字节数不等于图像宽度），导致像素偏移。使用以下正确的转换函数：

import 'package:image/image.dart' as imglib;

imglib.Image convertNV21ToImage(CameraImage cameraImage) {
  final yPlane = cameraImage.planes[0];
  final uvPlane = cameraImage.planes[1];
  
  final int originalWidth = cameraImage.width;
  final int originalHeight = cameraImage.height;
  
  final int yRowStride = yPlane.bytesPerRow;
  final int uvRowStride = uvPlane.bytesPerRow;
  final int uvPixelStride = uvPlane.bytesPerPixel!;
  
  final imageBuffer = imglib.Image(originalWidth, originalHeight);
  
  for (int y = 0; y < originalHeight; y++) {
    for (int x = 0; x < originalWidth; x++) {
      // 计算Y、UV分量的索引（必须用bytesPerRow而非图像宽度）
      final int yIndex = y * yRowStride + x;
      final int uvIndex = (y ~/ 2) * uvRowStride + (x ~/ 2) * uvPixelStride;
      
      final int yValue = yPlane.bytes[yIndex];
      final int uValue = uvPlane.bytes[uvIndex];
      final int vValue = uvPlane.bytes[uvIndex + 1];
      
      // NV21转RGB标准公式
      int r = (yValue + 1.370705 * (vValue - 128)).round();
      int g = (yValue - 0.698001 * (vValue - 128) - 0.337633 * (uValue - 128)).round();
      int b = (yValue + 1.732446 * (uValue - 128)).round();
      
      // 限制像素值在0-255范围内
      r = r.clamp(0, 255);
      g = g.clamp(0, 255);
      b = b.clamp(0, 255);
      
      imageBuffer.setPixelRgba(x, y, r, g, b, 255);
    }
  }
  
  return imageBuffer;
}

2. 旋转后同步转换人脸矩形坐标 #

旋转图像会互换宽高，原始人脸矩形的坐标无法直接用于裁剪。根据摄像头方向调整裁剪坐标：

// 转换NV21到imgLib.Image
imglib.Image? image = convertNV21ToImage(cameraImage);
final originalWidth = cameraImage.width;
final originalHeight = cameraImage.height;
imglib.Image? croppedFace;

if (camDirec == CameraLensDirection.front) {
  // 前置摄像头旋转270度，并转换坐标
  image = imglib.copyRotate(image!, angle: 270);
  final rotatedX = originalHeight - faceRect.top.toInt() - faceRect.height.toInt();
  final rotatedY = faceRect.left.toInt();
  final rotatedWidth = faceRect.height.toInt();
  final rotatedHeight = faceRect.width.toInt();
  croppedFace = imglib.copyCrop(image, x: rotatedX, y: rotatedY, width: rotatedWidth, height: rotatedHeight);
  // 前置摄像头需镜像翻转（可选，根据需求）
  croppedFace = imglib.copyFlipHorizontal(croppedFace);
} else {
  // 后置摄像头旋转90度，并转换坐标
  image = imglib.copyRotate(image!, angle: 90);
  final rotatedX = faceRect.top.toInt();
  final rotatedY = originalWidth - faceRect.left.toInt() - faceRect.width.toInt();
  final rotatedWidth = faceRect.height.toInt();
  final rotatedHeight = faceRect.width.toInt();
  croppedFace = imglib.copyCrop(image, x: rotatedX, y: rotatedY, width: rotatedWidth, height: rotatedHeight);
}

3. 验证每一步的图像正确性 #

• 先显示转换后的原始image，确认与相机预览一致
• 显示旋转后的image，确认无拉伸或偏移
• 最后显示裁剪后的croppedFace，验证人脸区域正确

最终展示代码 #

Image.memory(
  Uint8List.fromList(imglib.encodeBmp(croppedFace!)),
  width: 200,
  height: 200,
)

内容的提问来源于stack exchange，提问作者Naeem Mujeeb