Android设备GPU无法创建TFLite模型解释器，GPU与CPU算子支持是否一致？

问题描述 #

在Android设备上通过Flutter的tflite_flutter包运行TFLite模型时，CPU模式可正常运行，但GPU模式下无法创建解释器，报错信息如下：

I/tflite ( 8113): Created TensorFlow Lite delegate for NNAPI.
I/flutter ( 8113): Unable to create interpreter, Caught Exception: Invalid argument(s): Unable to create interpreter.

报错堆栈信息：

E/flutter ( 8113): #0      Classifier.interpreter (package:ace_example/tflite/classifier.dart)
E/flutter ( 8113): #1      _MyCameraPageState.onLatestImageAvailable (package:ace_example/camera/view.dart:193:34)
E/flutter ( 8113): #2      CameraController.startImageStream.<anonymous closure> (package:camera/src/camera_controller.dart:442:20)
E/flutter ( 8113): #3      _rootRunUnary (dart:async/zone.dart:1434:47)
E/flutter ( 8113): #4      _CustomZone.runUnary (dart:async/zone.dart:1335:19)
E/flutter ( 8113): #5      _CustomZone.runUnaryGuarded (dart:async/zone.dart:1244:7)
E/flutter ( 8113): #6      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341:11)
E/flutter ( 8113): #7      _DelayedData.perform (dart:async/stream_impl.dart:591:14)
E/flutter ( 8113): #8      _StreamImplEvents.handleNext (dart:async/stream_impl.dart:706:11)
E/flutter ( 8113): #9      _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:663:7)
E/flutter ( 8113): #10     _rootRun (dart:async/zone.dart:1418:47)
E/flutter ( 8113): #11     _CustomZone.run (dart:async/zone.dart:1328:19)
E/flutter ( 8113): #12     _CustomZone.runGuarded (dart:async/zone.dart:1236:7)
E/flutter ( 8113): #13     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1276:23)
E/flutter ( 8113): #14     _rootRun (dart:async/zone.dart:1426:13)
E/flutter ( 8113): #15     _CustomZone.run (dart:async/zone.dart:1328:19)
E/flutter ( 8113): #16     _CustomZone.runGuarded (dart:async/zone.dart:1236:7)
E/flutter ( 8113): #17     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1276:23)
E/flutter ( 8113): #18     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
E/flutter ( 8113): #19     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
E/flutter ( 8113): 
I/ech.ace_exampl( 8113): Background concurrent copying GC freed 578(97KB) AllocSpace objects, 21(2520KB) LOS objects, 49% free, 2453KB/4907KB, paused 5.309ms total 31.741ms

尝试使用GPUDelegateV2初始化解释器，代码如下，但问题仍未解决：

if (Platform.isAndroid) {
      final gpuDelegateV2 = GpuDelegateV2(
          options: GpuDelegateOptionsV2(
            isPrecisionLossAllowed: false,
            // TfLiteGpuInferenceUsage.fastSingleAnswer,
            inferencePriority1: TfLiteGpuInferencePriority.minLatency,
            // TfLiteGpuInferencePriority.auto,
            // TfLiteGpuInferencePriority.auto,
          ));
      _interpreterOptions = InterpreterOptions()..addDelegate(gpuDelegateV2);
      
      // InterpreterOptions()..useNnApiForAndroid = true;
      // _interpreterOptions = InterpreterOptions()..threads = 8;
    }

核心疑问：GPU支持的TFLite模型算子集合是否比CPU小？

回答 #

是的，TFLite GPU Delegate支持的算子集合确实比CPU解释器小。

CPU解释器是通用的软件实现，能运行所有TFLite定义的算子；而GPU Delegate是基于硬件加速的实现，只针对适合并行计算的常见算子（比如卷积、池化、矩阵乘法等）做了优化支持。对于小众算子、自定义算子，或是不适合GPU并行处理的特定序列类算子，GPU Delegate并没有对应的加速实现。

你的情况大概率是模型包含了GPU不支持的算子，导致解释器初始化失败。可以试试以下方案：

• 用TFLite Model Analyzer扫描模型，排查出不被GPU Delegate支持的算子
• 开启GPU Delegate的CPU fallback功能，让不支持的算子自动切换到CPU运行（需要在初始化GpuDelegateV2时配置对应选项）
• 采用混合模式：GPU处理支持的算子，CPU处理剩余部分

内容的提问来源于stack exchange，提问作者Ross