使用CameraController.startImageStream替代takePicture时encodeJpg索引越界求助

我们希望用CameraController.startImageStream()替代CameraController.takePicture()，获取和takePicture等价的返回值。试过现有方案但未成功，目前实现中调用encodeJpg()时出现索引越界错误，需要支持Android和iOS双平台，且追求更快的运行速度，可使用Flutter纯Dart或C++结合FFI实现。

当前实现代码 #

import 'package:image/image.dart' as imglib;

final cameras = await availableCameras();

final firstCamera = cameras.first;

cameraController = CameraController(
  firstCamera,
  ResolutionPreset.medium,
//  imageFormatGroup: ImageFormatGroup.yuv420,
);

await cameraController!.initialize();

await cameraController!.startImageStream((CameraImage cameraImage) async {
      cameraController!.stopImageStream();

      Uint8List list = cameraImage.planes.first.bytes;
      final width = cameraImage.width;
      final height = cameraImage.height;
      imglib.Image test = imglib.Image.fromBytes(width, height, list);

      List<int> _imageBytes = imglib.encodeJpg(test);
      Uint8List convert = Uint8List.fromList(_imageBytes);
      File file = File.fromRawPath(convert);
      XFile? result = XFile(file.path);
});

错误截图：

环境信息 #

fvm flutter doctor -v
[✓] Flutter (Channel stable, 3.3.6, on macOS 12.6 21G115 darwin-x64, locale en-JP)
    • Flutter version 3.3.6 on channel stable at /Users/xxxxx/fvm/versions/stable
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 6928314d50 (3 weeks ago), 2022-10-25 16:34:41 -0400
    • Engine revision 3ad69d7be3
    • Dart version 2.18.2
    • DevTools version 2.15.0

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/xxxxx/src/android-sdks
    • Platform android-33, build-tools 33.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13F100
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] VS Code (version 1.73.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.52.0

[✓] Connected device (3 available)
    • SHV43 (mobile)  • XXXXXXXXXXXXXXX • android-arm64  • Android 10 (API 29)
    • macOS (desktop) • macos           • darwin-x64     • macOS 12.6 21G115 darwin-x64
    • Chrome (web)    • chrome          • web-javascript • Google Chrome 107.0.5304.110

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

核心问题分析 #

直接取cameraImage.planes.first.bytes当作RGB数据传入imglib.Image.fromBytes是错误的：CameraImage默认返回YUV格式图像数据，而非RGB。YUV数据的平面结构、字节长度与RGB完全不同，直接解析必然导致索引越界。

修复步骤（Flutter纯Dart实现） #

1. 固定图像格式为YUV420 #

初始化CameraController时必须指定统一格式，确保双平台返回一致的YUV数据：

cameraController = CameraController(
  firstCamera,
  ResolutionPreset.medium,
  imageFormatGroup: ImageFormatGroup.yuv420, // 必须启用
);

2. 正确转换YUV到RGB #

使用image库的YUV转RGB逻辑，针对YUV420格式处理三个平面的数据：

import 'package:image/image.dart' as imglib;

imglib.Image convertYUV420toImage(CameraImage cameraImage) {
  final int width = cameraImage.width;
  final int height = cameraImage.height;

  final int uvRowStride = cameraImage.planes[1].bytesPerRow;
  final int uvPixelStride = cameraImage.planes[1].bytesPerPixel!;

  final imglib.Image image = imglib.Image(width, height);

  for (int w = 0; w < width; w++) {
    for (int h = 0; h < height; h++) {
      final int uvIndex = uvPixelStride * (w ~/ 2) + uvRowStride * (h ~/ 2);
      final int index = h * width + w;

      final y = cameraImage.planes[0].bytes[index];
      final u = cameraImage.planes[1].bytes[uvIndex];
      final v = cameraImage.planes[2].bytes[uvIndex];

      image.data[index] = imglib.YUV2RGB(y, u, v);
    }
  }
  return image;
}

3. 修改startImageStream逻辑 #

替换原图像解析代码，同时修正文件保存方式（File.fromRawPath不能直接使用Uint8List，需先写入临时文件）：

import 'package:path_provider/path_provider.dart';

await cameraController!.startImageStream((CameraImage cameraImage) async {
  await cameraController!.stopImageStream(); // 停止流避免重复回调

  // 转换YUV到RGB图像
  imglib.Image rgbImage = convertYUV420toImage(cameraImage);
  
  // 编码为JPG
  List<int> jpgBytes = imglib.encodeJpg(rgbImage);
  
  // 保存到临时文件，模拟takePicture的XFile输出
  final tempDir = await getTemporaryDirectory();
  final file = File('${tempDir.path}/${DateTime.now().millisecondsSinceEpoch}.jpg');
  await file.writeAsBytes(jpgBytes);
  
  XFile result = XFile(file.path);
  // 此处处理result，如上传或展示
});

性能优化方案（C++ FFI实现） #

若纯Dart转换速度不足，可通过C++实现YUV转RGB并通过FFI调用，速度可提升2-3倍：

1. 编写C++转换代码 #

#include <cstdint>

extern "C" {
void convertYUV420ToRGB(
    uint8_t* yPlane, uint8_t* uPlane, uint8_t* vPlane,
    int width, int height,
    int yRowStride, int uvRowStride, int uvPixelStride,
    uint8_t* outputRGB) {
    for (int h = 0; h < height; h++) {
        for (int w = 0; w < width; w++) {
            int uvIndex = uvPixelStride * (w / 2) + uvRowStride * (h / 2);
            int index = h * width + w;
            
            int y = yPlane[index];
            int u = uPlane[uvIndex] - 128;
            int v = vPlane[uvIndex] - 128;
            
            int r = y + (359 * v) / 256;
            int g = y - (88 * u + 183 * v) / 256;
            int b = y + (454 * u) / 256;
            
            r = r < 0 ? 0 : (r > 255 ? 255 : r);
            g = g < 0 ? 0 : (g > 255 ? 255 : g);
            b = b < 0 ? 0 : (b > 255 ? 255 : b);
            
            outputRGB[index * 3] = r;
            outputRGB[index * 3 + 1] = g;
            outputRGB[index * 3 + 2] = b;
        }
    }
}
}

将代码编译为对应平台的动态库后，在Flutter中通过FFI调用，替换Dart转换函数即可。

验证要点 #

• 确保Android和iOS的相机权限已在配置文件中声明
• 测试不同分辨率下的转换稳定性
• 对比takePicture的输出，确保图像质量、格式一致

内容的提问来源于stack exchange，提问作者Ganessa