tflite_flutter运行MediaPipe模型推理耗时过高求助

问题概述 #

• 同一MediaPipe TFLite模型在原生Android上推理仅需约90ms，但使用tflite_flutter库时耗时高达900ms
• 图像预处理仅耗时约70ms，排除预处理环节对耗时的影响
• 输入缓冲区格式为[1,256,256,3]，已尝试基础线程配置但未改善性能

相关代码 #

import 'dart:math';
import 'dart:typed_data';
import 'package:camera/camera.dart';
import 'package:camera_app/utils/image_utils.dart';
import 'package:flutter/cupertino.dart';
import 'package:image/image.dart' as image_lib;
import 'dart:io';
import 'package:tflite_flutter/tflite_flutter.dart';
import '../core/constants.dart';

class MediaPipeProcessor {
  late List<Object> inputs;
  List<dynamic> identity = Float32List(1*195).reshape([1,195]);
  List<dynamic> identity_1 = Float32List(1*1).reshape([1 , 1]);
  List<dynamic> identity_2 = Float32List(1*256*256*1).reshape([1,256,256,1]);
  List<dynamic> identity_3 = Float32List(1*64*64*39).reshape([1,64,64,39]);
  List<dynamic> identity_4 = Float32List(1*117).reshape([1,117]);
  late int imageWidth;
  late int imageHeight;
  late Interpreter _interpreter ;
  late List<dynamic> inputBuffer ;

  MediaPipeProcessor({Interpreter? interpreter}) {
    loadModel(interpreter: interpreter);
  }
  Interpreter get interpreter => _interpreter;

  loadModel({Interpreter? interpreter}) async {
    try {
      InterpreterOptions? interpreterOptions;
      //GPU DELEGATE CODE
      /* GPU delegate code
      if (Platform.isAndroid) {
        final gpuDelegateV2 = GpuDelegateV2(
          options: GpuDelegateOptionsV2(
            isPrecisionLossAllowed: false,
            inferencePreference: TfLiteGpuInferenceUsage.fastSingleAnswer,
            inferencePriority1: TfLiteGpuInferencePriority.minLatency,
            inferencePriority2: TfLiteGpuInferencePriority.auto,
            inferencePriority3: TfLiteGpuInferencePriority.auto,
          ),
        );

        interpreterOptions = InterpreterOptions()
          ..addDelegate(gpuDelegateV2)
          ..threads = 4;
      } else if (Platform.isIOS) {
        final gpuDelegateMetal = GpuDelegate(
          options: GpuDelegateOptions(
            allowPrecisionLoss: true,
            waitType: TFLGpuDelegateWaitType.active,
          ),
        );
        interpreterOptions = InterpreterOptions()
          ..addDelegate(gpuDelegateMetal)
          ..threads = 4;
      }
      */
      _interpreter = interpreter ??
          await Interpreter.fromAsset(
            "assets/mediapipe.tflite",
            options: InterpreterOptions()..threads = 4,
          );
      
    } catch (e) {
      debugPrint("Error while creating interpreter: $e");
    }
    //TESTING CODE
    /* Testing code
    var outputTensors = interpreter.getOutputTensors();
    var inputTensors = interpreter.getInputTensors();
    List<List<int>> _outputShapes = [];

    outputTensors.forEach((tensor) {
      print("Output Tensor: " + tensor.toString());
      _outputShapes.add(tensor.shape);
    });
    inputTensors.forEach((tensor) {
      print("Input Tensor: " + tensor.toString());
    });

    print("------------------[A}========================\n" +
        _outputShapes.toString());
    */
  }

  int frameNo = 0;

  runModel() async {
    Map<int, Object> outputs = {0 : identity , 1 : identity_1 , 2 : identity_2 , 3 : identity_3 , 4 : identity_4};
    final startTime = DateTime.now().microsecondsSinceEpoch;
    _interpreter.runForMultipleInputs(inputs, outputs);
    final endTime = DateTime.now().microsecondsSinceEpoch;
    final difference = endTime - startTime;
    print("Interpreter Execution Time : $difference ");
    //IT IS SHOWING AROUND 900MS 
    // print("📊 Output list prepared Mediapipe $identity");
  }

  Map<String, List<dynamic>> parseData() {
    List result = [];
    List identityT = identity.reshape([39 , 5]);
    for (var i = 0; i < 39 ; i ++) {
      double x = identityT[i][0] / 256 * imageWidth; // x-coordinate
      double y = identityT[i][1] / 256 * imageHeight; // y-coordinate
      // Actual Cordinates
      result.add([x,y]);
    }
    print("$result");
    return {
      'resultsCoordinates': result,
      'frames' : inputBuffer
    };
  }


  Future<void> performOperations(CameraImage cameraImage) async {
    image_lib.Image? convertedImage;
    convertedImage = ImageUtils.convertCameraImage(cameraImage);
    if (Platform.isAndroid) {
      // convertedImage = image_lib.flipHorizontal(convertedImage!);
    }
    // Manual image preprocessing
    imageWidth = convertedImage!.width;
    imageHeight = convertedImage.height;

    image_lib.Image resizedImage = image_lib.copyResize(
        convertedImage,
        width: 256,
        height: 256
    );

    // final int pixelCount = resizedImage.width * resizedImage.height;
    // inputBuffer = Float32List(pixelCount * 3);
    // const double normalizationFactor = 1.0 / 255.0;
    // for (int i = 0; i < pixelCount; i++) {
    //   final pixel = resizedImage[i];
    //   inputBuffer[i * 3] = ((pixel >> 16) & 0xFF) * normalizationFactor; // Red
    //   inputBuffer[i * 3 + 1] = ((pixel >> 8) & 0xFF) * normalizationFactor; // Green
    //   inputBuffer[i * 3 + 2] = (pixel & 0xFF) * normalizationFactor; // Blue
    // }

    final pixels = resizedImage.getBytes(format: image_lib.Format.rgb);
    inputBuffer = Float32List.fromList(pixels.map((pixel) => pixel / 255.0).toList());
    inputBuffer = inputBuffer.reshape([1 , 256 , 256 , 3 ]) ;
    inputs = [inputBuffer];
    // print("🖼️ Image resized and normalized Mediapipe: $inputBuffer");
  }
}

优化方案 #

1. 启用GPU加速Delegate #

原生Android的低耗时核心是使用了GPU加速，而当前Flutter代码默认用CPU推理，这是耗时差距的主要原因。恢复并优化GPU Delegate配置：

• Android端使用GpuDelegateV2，优先配置低延迟推理参数
• iOS端使用GpuDelegate，允许精度损失（若模型兼容）以提升速度
• 注意：Delegate资源需在Interpreter销毁时手动释放，避免内存泄漏

2. 优化Interpreter初始化 #

• 确保Interpreter全局单例初始化，避免重复加载模型的开销
• Android端启用NNAPI，让系统自动选择最优硬件加速（CPU/GPU/NPU）
• 线程数设置为设备核心数，避免线程切换浪费资源

修改后的loadModel方法示例：

loadModel({Interpreter? interpreter}) async {
  try {
    InterpreterOptions interpreterOptions = InterpreterOptions();
    // Android启用NNAPI
    if (Platform.isAndroid) {
      interpreterOptions.useNNAPI = true;
      final gpuDelegateV2 = GpuDelegateV2(
        options: GpuDelegateOptionsV2(
          isPrecisionLossAllowed: false,
          inferencePreference: TfLiteGpuInferenceUsage.fastSingleAnswer,
          inferencePriority1: TfLiteGpuInferencePriority.minLatency,
          inferencePriority2: TfLiteGpuInferencePriority.auto,
          inferencePriority3: TfLiteGpuInferencePriority.auto,
        ),
      );
      interpreterOptions.addDelegate(gpuDelegateV2);
    } else if (Platform.isIOS) {
      final gpuDelegateMetal = GpuDelegate(
        options: GpuDelegateOptions(
          allowPrecisionLoss: true,
          waitType: TFLGpuDelegateWaitType.active,
        ),
      );
      interpreterOptions.addDelegate(gpuDelegateMetal);
    }
    // 设置线程数为设备核心数
    interpreterOptions.threads = Platform.numberOfProcessors;

    _interpreter = interpreter ??
        await Interpreter.fromAsset(
          "assets/mediapipe.tflite",
          options: interpreterOptions,
        );
  } catch (e) {
    debugPrint("Error while creating interpreter: $e");
  }
}

3. 输入缓冲区优化 #

避免动态reshape操作带来的开销，直接创建符合模型要求的连续内存缓冲区：

// 优化输入缓冲区创建
final pixelCount = 256 * 256 * 3;
inputBuffer = Float32List(pixelCount);
final pixels = resizedImage.getBytes(format: image_lib.Format.rgb);
for (int i = 0; i < pixelCount; i++) {
  inputBuffer[i] = pixels[i] / 255.0;
}
// 直接传入符合形状的张量
inputs = [inputBuffer.reshape([1, 256, 256, 3])];

内容的提问来源于stack exchange，提问作者KARAN CHOUHAN