GPT2转TensorFlow Lite官方示例失败，无法获取Android可用模型

问题详情 #

• 按照TensorFlow官方生成式AI Android示例仓库步骤操作时，触发报错：NotImplementedError: Cannot convert a symbolic tf.Tensor (StatefulPartitionedCall:1) to a numpy array
• 初始在Windows环境操作，发现keras_nlp仅支持Debian系统，转至Debian虚拟机测试
• 尝试将keras、keras_nlp、tensorflow降级至2023年5月仓库README更新前的版本，报错依旧

核心需求 #

获取Android端可用的autocomplete.tflite文件：

• 该模型无法像BERT QA、Smart Reply等示例直接下载
• Gradle下载的AAR包中不包含此模型
• Android示例代码加载模型时，在val descriptor = context.assets.openFd(TFLITE_MODEL)处报错

执行的官方步骤代码 #

import keras_nlp
import tensorflow as tf

gpt2_tokenizer = keras_nlp.models.GPT2Tokenizer.from_preset("gpt2_base_en")

gpt2_preprocessor = keras_nlp.models.GPT2CausalLMPreprocessor.from_preset(
  "gpt2_base_en",
  sequence_length=256,
  add_end_token=True,
)

gpt2_lm = keras_nlp.models.GPT2CausalLM.from_preset(
  "gpt2_base_en", 
  preprocessor=gpt2_preprocessor,
)

@tf.function
def generate(prompt, max_length):
  # prompt: 输入到大语言模型的字符串格式提示词
  # max_length: 生成token的最大长度
  return gpt2_lm.generate(prompt, max_length)
concrete_func = generate.get_concrete_function(tf.TensorSpec([], tf.string), 100)

def run_inference(input, generate_tflite):
  interp = interpreter.InterpreterWithCustomOps(
    model_content=generate_tflite,
    custom_op_registerers=tf_text.tflite_registrar.SELECT_TFTEXT_OPS)
  interp.get_signature_list()

  generator = interp.get_signature_runner('serving_default')
  output = generator(prompt=np.array([input]))

gpt2_lm.jit_compile = False
converter = tf.lite.TFLiteConverter.from_concrete_functions(
  [concrete_func],
  gpt2_lm)

converter.target_spec.supported_ops = [
  tf.lite.OpsSet.TFLITE_BUILTINS, # 启用TFLite内置算子
  tf.lite.OpsSet.SELECT_TF_OPS, # 启用TF算子
]
converter.allow_custom_ops = True
converter.target_spec.experimental_select_user_tf_ops = [
  "UnsortedSegmentJoin",
  "UpperBound"
]
converter._experimental_guarantee_all_funcs_one_use = True
generate_tflite = converter.convert()
run_inference("I'm enjoying a", generate_tflite)

内容的提问来源于stack exchange，提问作者dobre dani