使用tflite_flutter做图像分类时interpreter.run类型不匹配报错求助

错误信息 #

首次运行时出现类型转换异常：

E/Parcel  (29971): Reading a NULL string not supported here.
I/flutter (29971): afah aa
I/flutter (29971): afah ab
E/flutter (29971): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: type 'List<double>' is not a subtype of type 'List<int>' of 'value'
E/flutter (29971): #0      List.[]= (dart:core-patch/growable_array.dart)
E/flutter (29971): #1      Tensor._duplicateList (package:tflite_flutter/src/tensor.dart:236:10)
E/flutter (29971): #2      Tensor.copyTo (package:tflite_flutter/src/tensor.dart:202:7)
E/flutter (29971): #3      Interpreter.runForMultipleInputs (package:tflite_flutter/src/interpreter.dart:183:24)
E/flutter (29971): #4      Interpreter.run (package:tflite_flutter/src/interpreter.dart:172:5)
E/flutter (29971): #5      _HomePage.runInference (package:thesis_app/homepage.dart:119:17)
E/flutter (29971): #6      _HomePage.processImage (package:thesis_app/homepage.dart:104:7)
E/flutter (29971): #7      _HomePage.openGallery.<anonymous closure> (package:thesis_app/homepage.dart:282:7)
E/flutter (29971): #8      State.setState (package:flutter/src/widgets/framework.dart:1139:30)
E/flutter (29971): #9      _HomePage.openGallery (package:thesis_app/homepage.dart:281:5)
E/flutter (29971): <asynchronous suspension>

尝试将输入强制转换为List<List<List<int>>>后，出现新异常：

E/flutter (29971): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: type 'List<List<List<List<num>>>>' is not a subtype of type 'List<List<List<int>>>' in type cast

问题代码 #

// Load model
Future<void> loadModel() async {
  final options = InterpreterOptions();

  // Use XNNPACK Delegate
  if (Platform.isAndroid) {
    options.addDelegate(XNNPackDelegate());
  }

  if (Platform.isIOS) {
    options.addDelegate(GpuDelegate());
  }

  // Load model from assets
  interpreter = await Interpreter.fromAsset(modelPath, options: options);
  // Get tensor input shape [1, 224, 224, 3]
  inputTensor = interpreter.getInputTensors().first;
  // Get tensor output shape [1, 1001]
  outputTensor = interpreter.getOutputTensors().first;
  setState(() {});

  log('Interpreter loaded successfully');
}

// Load labels from assets
Future<void> loadLabels() async {
  final labelTxt = await rootBundle.loadString(labelsPath);
  labels = labelTxt.split('\n');
}

Future<void> processImage() async {
  if (imagePath != null) {
    // Read image bytes from file
    final imageData = File(imagePath!).readAsBytesSync();

    // Decode image using package:image/image.dart
    image = img.decodeImage(imageData);
    setState(() {});

    // Resize image for model input (Mobilenet use [224, 224])
    final imageInput = img.copyResize(
      image!,
      width: 224,
      height: 224,
    );

    // Get image matrix representation [224, 224, 3]
    final imageMatrix = List.generate(
      imageInput.height,
      (y) => List.generate(
        imageInput.width,
        (x) {
          final pixel = imageInput.getPixel(x, y);
          return [pixel.r, pixel.g, pixel.b];
        },
      ),
    );

    // Run model inference
    runInference(imageMatrix);
  }
}

// Run inference
Future<void> runInference(
  List<List<List<num>>> imageMatrix,
) async {
  print("afah aa");
  // Set tensor input [1, 224, 224, 3]
  final input = [imageMatrix];
  // Set tensor output [1, 1001]
  final output = [List<int>.filled(6, 1)];
  print("afah ab");
  // Run inference
  interpreter.run(input, output);
  print("afah ac");
  // Get first output tensor
  final result = output.first;
  print("afah ad");
  // Set classification map {label: points}
  classification = <String, int>{};

  print("afah af");

  for (var i = 0; i < result.length; i++) {
    if (result[i] != 0) {
      // Set label: points
      classification![labels[i]] = result[i];
    }
  }

  setState(() {});
}

解决方案 #

问题根源在于输入数据类型与模型要求不匹配，以及输出张量长度错误，修正步骤如下：

1. 调整图像预处理，转为模型要求的float32格式 #

Mobilenet模型要求输入为归一化到0-1之间的float类型数据，修改processImage中的imageMatrix生成逻辑：

final imageMatrix = List.generate(
  imageInput.height,
  (y) => List.generate(
    imageInput.width,
    (x) {
      final pixel = imageInput.getPixel(x, y);
      // 将0-255的int像素值归一化为0-1的double
      return [pixel.r / 255.0, pixel.g / 255.0, pixel.b / 255.0];
    },
  ),
);

2. 修正输出张量的长度和类型 #

模型输出形状为[1,1001]，且输出是概率值（float类型），修改runInference中的输出数组：

// 匹配模型输出形状[1,1001]，创建double类型数组
final output = [List<double>.filled(1001, 0.0)];

3. 调整runInference的参数类型和后续处理 #

更新参数类型为List<List<List<double>>>，并适配double类型的输出结果：

Future<void> runInference(
  List<List<List<double>>> imageMatrix,
) async {
  print("afah aa");
  final input = [imageMatrix];
  final output = [List<double>.filled(1001, 0.0)];
  print("afah ab");
  interpreter.run(input, output);
  print("afah ac");
  final result = output.first;
  print("afah ad");
  classification = <String, double>{};

  print("afah af");

  for (var i = 0; i < result.length; i++) {
    // 过滤低概率结果
    if (result[i] > 0.01) {
      classification![labels[i]] = result[i];
    }
  }

  setState(() {});
}

额外优化（可选） #

为提升性能，可将图像转为一维Float32数组：

final input = Float32List(224 * 224 * 3);
int index = 0;
for (int y = 0; y < 224; y++) {
  for (int x = 0; x < 224; x++) {
    final pixel = imageInput.getPixel(x, y);
    input[index++] = pixel.r / 255.0;
    input[index++] = pixel.g / 255.0;
    input[index++] = pixel.b / 255.0;
  }
}
// 输入传入一维数组包裹的列表
interpreter.run([input], output);

内容的提问来源于stack exchange，提问作者Ali Mahfud