自定义TFLite模型检测对象时触发PlatformException求助

我用Cloud AutoML训练了一个检测纸张标记的自定义模型，导出为TFLite文件并托管在Firebase上。模型下载和ObjectDetector初始化都成功了，但处理输入图像时触发错误。

代码实现 #

Cubit中初始化检测器 #

initialiseDetector({double confidenceThreshold = 0.5, int maximumLabelsPerObject = 10}) async {
  emit(ShoddyLoading(state.mainShoddyState.copyWith(message: 'Loading object detector')));
  try {
    ObjectDetector objectDetector = await ShoddyHelper.initialiseDetector(
      processingFromDownloadedFile: true,
      modelFile: state.mainShoddyState.modelFile,
      confidenceThreshold: confidenceThreshold,
      maximumLabelsPerObject: maximumLabelsPerObject,
    );
    emit(ShoddyModelLoaded(state.mainShoddyState.copyWith(objectDetector: objectDetector, message: 'Ready to start processing images')));
  } catch (error) {
    emit(ShoddyError(state.mainShoddyState.copyWith(message: error.toString())));
  }
}

模型下载与加载工具类 #

static Future<ObjectDetector> initialiseDetector({File? modelFile, bool processingFromDownloadedFile = true, required double confidenceThreshold, required int maximumLabelsPerObject}) async {
  if (processingFromDownloadedFile) {
    if (modelFile != null) {
      return await initializeLocalDetector(modelFile, confidenceThreshold, maximumLabelsPerObject);
    } else {
      File modelFile = await loadModelFileFromFirebase();
      return await initializeLocalDetector(modelFile, confidenceThreshold, maximumLabelsPerObject);
    }
  } else {
    return await initializeFirebaseDetector(confidenceThreshold, maximumLabelsPerObject);
  }
}

// Download the model file from firebase first
static Future<File> loadModelFileFromFirebase(String modelName) async {
  try {
    FirebaseModelDownloader downloader = FirebaseModelDownloader.instance;

    List<FirebaseCustomModel> models = await downloader.listDownloadedModels();
    for (FirebaseCustomModel model in models) {
      print('Name: ${model.name}');
    }

    FirebaseModelDownloadConditions conditions = FirebaseModelDownloadConditions(
      iosAllowsCellularAccess: true,
      iosAllowsBackgroundDownloading: false,
      androidChargingRequired: false,
      androidWifiRequired: false,
      androidDeviceIdleRequired: false,
    );

    FirebaseCustomModel model = await downloader.getModel(
      modelName,
      FirebaseModelDownloadType.latestModel,
      conditions,
    );

    File modelFile = model.file;

    return modelFile;
  } catch (exception) {
    print('Failed on loading your model from Firebase: $exception');
    print('The program will not be resumed');
    rethrow;
  }
}

// Use a file downloaded from firebase
static Future<ObjectDetector> initializeLocalDetector(File modelFile, double confidenceThreshold, int maximumLabelsPerObject) async {
  try {
    final options = LocalObjectDetectorOptions(
      mode: DetectionMode.single,
      modelPath: modelFile.path,
      classifyObjects: true,
      multipleObjects: true,
      confidenceThreshold: confidenceThreshold,
      maximumLabelsPerObject: maximumLabelsPerObject,
    );

    return ObjectDetector(options: options);
  } catch (exception) {
    print('Failed on loading your model to the TFLite interpreter: $exception');
    print('The program will not be resumed');
    rethrow;
  }
}

// Use the model file directly from firebase
static Future<ObjectDetector> initializeFirebaseDetector(String modelName, double confidenceThreshold, int maximumLabelsPerObject) async {
  try {
    final options = FirebaseObjectDetectorOptions(
      mode: DetectionMode.single,
      modelName: modelName,
      classifyObjects: true,
      multipleObjects: true,
      confidenceThreshold: confidenceThreshold,
      maximumLabelsPerObject: maximumLabelsPerObject,
    );

    return ObjectDetector(options: options);
  } catch (exception) {
    print('Failed on loading your model to the TFLite interpreter: $exception');
    print('The program will not be resumed');
    rethrow;
  }
}

图像处理函数 #

processImage(File file) async {
  emit(ShoddyModelProcessing(state.mainShoddyState.copyWith(message: 'Looking for objects on the selected image')));
  try {
    List<dynamic>? results = [];
    if (state.mainShoddyState.objectDetector != null) {
      InputImage inputImage = InputImage.fromFilePath(file.path);
      List<DetectedObject> objects = await state.mainShoddyState.objectDetector!.processImage(inputImage);
      if (objects.isNotEmpty) {
        List<ObjectModel> objects = results.map((result) => ObjectModel(result)).toList();
        emit(ShoddyModelProcessed(state.mainShoddyState.copyWith(objects: objects, filteredObjects: objects, message: 'Image processed with results')));
        changeMatchPercentage(0.35);
      } else {
        emit(ShoddyModelProcessed(state.mainShoddyState.copyWith(objects: [], filteredObjects: [], message: 'Image processed with no results')));
      }
    }
  } catch (error) {
    emit(ShoddyError(state.mainShoddyState.copyWith(message: error.toString())));
  }
}

触发错误的代码行 #

List<DetectedObject> objects = await state.mainShoddyState.objectDetector!.processImage(inputImage);

错误信息 #

PlatformException(Error 3, com.google.visionkit.pipeline.error, Pipeline failed to fully start:
CalculatorGraph::Run() failed in Run:
Calculator::Open() for node "BoxClassifierCalculator" failed: #vk Unexpected number of dimensions for output index 0: got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1)., null)

这个错误的核心是模型输出格式与Flutter ObjectDetector的预期不匹配。Cloud AutoML导出的TFLite目标检测模型，默认输出可能是3D张量，但Flutter的ObjectDetector（基于Google Vision Kit）要求分类器输出为2D（BxN，B=1）或4D（BxHxWxN，其中H=1、W=1、B=1）格式。

具体修复步骤： #

• 重新导出Cloud AutoML模型
  在导出TFLite模型时，确保选择与Vision Kit兼容的输出格式。在Cloud AutoML导出界面，选择“适用于移动设备的TensorFlow Lite”，并启用“与ML Kit兼容”选项；如果没有该选项，需用TensorFlow Lite Converter重新转换模型，指定输出为2D或符合要求的4D张量，或调整模型头部结构适配维度要求。

• 调整Detector配置参数
  你初始化LocalObjectDetectorOptions时设置了classifyObjects: true，如果模型本身没有分类分支或分支输出格式不符，会触发错误：

  • 若仅需目标检测，将classifyObjects改为false可直接绕过错误；
  • 若需要分类功能，必须确保模型分类输出维度符合Vision Kit要求。

• 验证模型输入输出格式
  使用TensorFlow Lite Inspector工具查看模型张量维度：

  tflite_inspect model --model_path=your_model.tflite
  

  确认输出张量为2D或指定4D格式，若为3D则需重新处理模型。

• 修复代码中的变量重定义问题
  processImage函数里存在变量重定义错误，将：

  List<ObjectModel> objects = results.map((result) => ObjectModel(result)).toList();
  

  修改为：

  List<ObjectModel> objectModels = objects.map((obj) => ObjectModel(obj)).toList();
  

  同时results为空列表，需直接用检测到的objects进行转换。

内容的提问来源于stack exchange，提问作者Theuno de Bruin