保存Keras集成模型时报ValueError: Structure is a scalar错误如何解决

运行环境：Python 3.7，keras2.3.1，tensorflow2.0.0
需要实现的需求：将两个已训练好的Keras模型集成后保存为新的模型，构建模型代码如下：

text_in = tensorflow.keras.layers.Input(shape=(MAX_SEQ_LEN,), dtype="int32", name='myinput')
mods = [model_0, model_1]
model_outputs = [model(text_in) for model in mods]
ensemble_output = tf.keras.layers.Average()(model_outputs)
ensemble_model = tf.keras.Model(inputs=text_in, outputs=ensemble_output)

执行ensemble_model.save(fname)时报错如下：

ValueError                                Traceback (most recent call last)
<ipython-input-65-0212c05a2372> in <module>
      1 fname = 'model_multiclass_multilabel_{}.h5'.format(datetime.utcnow())
      2 base = fname.split('.h5')[0]
----> 3 model.save(fname)

/opt/miniconda3/envs/ml/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py in save(self, filepath, overwrite, include_optimizer, save_format, signatures, options)
    973     """
    974     saving.save_model(self, filepath, overwrite, include_optimizer, save_format,
--> 975                       signatures, options)
    976 
    977   def save_weights(self, filepath, overwrite=True, save_format=None):

/opt/miniconda3/envs/ml/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/save.py in save_model(model, filepath, overwrite, include_optimizer, save_format, signatures, options)
    110           'or using `save_weights`.')
    111     hdf5_format.save_model_to_hdf5(
--> 112         model, filepath, overwrite, include_optimizer)
    113   else:
    114     saved_model_save.save(model, filepath, overwrite, include_optimizer,

/opt/miniconda3/envs/ml/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/hdf5_format.py in save_model_to_hdf5(model, filepath, overwrite, include_optimizer)
     97 
     98   try:
--- 99     model_metadata = saving_utils.model_metadata(model, include_optimizer)
    100     for k, v in model_metadata.items():
    101       if isinstance(v, (dict, list, tuple)):

/opt/miniconda3/envs/ml/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/saving_utils.py in model_metadata(model, include_optimizer, require_config)
    160   model_config = {'class_name': model.__class__.__name__}
    161   try:
--> 162     model_config['config'] = model.get_config()
    163   except NotImplementedError as e:
    164     if require_config:

/opt/miniconda3/envs/ml/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py in get_config(self)
    885     if not self._is_graph_network:
    886       raise NotImplementedError
--> 887     return copy.deepcopy(get_network_config(self))
    888 
    889   @classmethod

/opt/miniconda3/envs/ml/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py in get_network_config(network, serialize_layer_fn)
   1940           filtered_inbound_nodes.append(node_data)
   1941 
-> 1942     layer_config = serialize_layer_fn(layer)
   1943     layer_config['name'] = layer.name
   1944     layer_config['inbound_nodes'] = filtered_inbound_nodes

/opt/miniconda3/envs/ml/lib/python3.7/site-packages/tensorflow_core/python/keras/utils/generic_utils.py in serialize_keras_object(instance)
    138   if hasattr(instance, 'get_config'):
    139     return serialize_keras_class_and_config(instance.__class__.__name__,
--> 140                                             instance.get_config())
    141   if hasattr(instance, '__name__'):
    142     return instance.__name__

/opt/miniconda3/envs/ml/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py in get_config(self)
    885     if not self._is_graph_network:
    886       raise NotImplementedError
--> 887     return copy.deepcopy(get_network_config(self))
    888 
    889   @classmethod

/opt/miniconda3/envs/ml/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py in get_network_config(network, serialize_layer_fn)
   1956     model_inputs.append(
   1957         tf_utils.ListWrapper([layer.name, new_node_index, tensor_index]))
-> 1958   model_inputs = nest.pack_sequence_as(network._nested_inputs, model_inputs)
   1959   # Preserve external Keras compat for Models with single input.
   1960   if not nest.is_sequence(model_inputs):

/opt/miniconda3/envs/ml/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py in pack_sequence_as(structure, flat_sequence, expand_composites)
    454     if len(flat_sequence) != 1:
    455       raise ValueError("Structure is a scalar but len(flat_sequence) == %d > 1"
--> 456                        % len(flat_sequence))
    457     return flat_sequence[0]
    458 

ValueError: Structure is a scalar but len(flat_sequence) == 0 > 1

已确认子模型训练使用的Keras/TF版本和当前环境一致，输入类型完全匹配。

该错误是TensorFlow 2.0.0版本的已知缺陷：h5格式的序列化逻辑无法正确处理将多个预训练Keras子模型作为层嵌套进新模型的场景，识别子模型输入节点结构时出现判断错误，导致配置构建失败。

• 方案一：使用TensorFlow原生SavedModel格式保存（优先推荐）
  不需要修改任何模型结构，仅需要调整保存时的格式参数，该格式对嵌套模型的兼容性远高于h5：

  # 保存模型，会生成一个名为ensemble_model的文件夹
  ensemble_model.save("ensemble_model", save_format="tf")
  # 加载模型
  ensemble_model = tf.keras.models.load_model("ensemble_model")
  

• 方案二：如果必须使用h5格式存储，采用权重单独保存的方案
  跳过模型结构的序列化步骤，仅保存权重，使用时先重建模型结构再加载权重，兼容性最高：

  # 保存权重
  ensemble_model.save_weights("ensemble_weights.h5")
  
  # 使用时先按照你原来的代码重新构建ensemble_model的结构，再执行加载
  ensemble_model.load_weights("ensemble_weights.h5")
  

• 方案三：排查子模型加载逻辑
  确认两个子模型都是通过tf.keras.models.load_model加载，不要混用独立Keras库的加载接口，否则会导致子模型的图网络标记异常，触发序列化错误。

内容的提问来源于stack exchange，提问作者Steven