TensorFlowJS加载Mask RCNN模型遇int32与float32类型不匹配错误

我基于Mask R-CNN Inception ResNet V2 1024x1024做迁移学习训练了目标检测模型，转成JS版本后运行报错：ERROR provided in model.execute(dict) must be int32, but was float32。

操作步骤 #

• 1. 生成training.json、validation.json、testing.json标注文件及label_map.txt，将图像预处理为1024*1024尺寸。
• 2. 修改create_coco_tf_record.py中的include_masks参数为True，生成tfrecord文件：

tf.flags.DEFINE_boolean(
    'include_masks', True, 'Whether to include instance segmentations masks '
)

执行命令：

python create_coco_tf_record.py ^
--logtostderr ^
--train_image_dir=C:/model/ai_container/training ^
--val_image_dir=C:/model/ai_container/vidation ^
--test_image_dir=C:/model/ai_container/testing ^
--train_annotations_file=C:/model/ai_container/training/training.json ^
--val_annotations_file=C:/model/ai_container/validation/coco_validation.json ^
--testdev_annotations_file=C:/model/ai_container/testing/coco_testing.json ^
--output_dir=C:/model/ai_container/tfrecord

• 3. 修改基础mask-rcnn配置文件，将batch和num_steps设为1，关键配置片段：

train_config: {
  batch_size: 1
  num_steps: 1
  optimizer {
    momentum_optimizer: {
      learning_rate: {
        cosine_decay_learning_rate {
          learning_rate_base: 0.008
          total_steps: 200000
          warmup_learning_rate: 0.0
          warmup_steps: 5000
        }
      }
      momentum_optimizer_value: 0.9
    }
    use_moving_average: false
  }
  gradient_clipping_by_norm: 10.0
  fine_tune_checkpoint_version: V2
  fine_tune_checkpoint: "C:/ObjectDetectionAPI/mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8/checkpoint/ckpt-0"
  fine_tune_checkpoint_type: "detection"
  data_augmentation_options {
    random_horizontal_flip {
    }
  }
}

train_input_reader: {
  label_map_path: "C:/model/ai_container/label_map.txt"
  tf_record_input_reader {
    input_path: "C:/model/ai_container/tfrecord/coco_train.record*"
  }
  load_instance_masks: true
  mask_type: PNG_MASKS
}

eval_config: {
  metrics_set: "coco_detection_metrics"
  metrics_set: "coco_mask_metrics"
  eval_instance_masks: true
  use_moving_averages: false
  batch_size: 1
  include_metrics_per_category: false
}

eval_input_reader: {
  label_map_path: "C:/model/ai_container/label_map.txt"
  shuffle: false
  num_epochs: 1
  tf_record_input_reader {
    input_path: "C:/model/ai_container/tfrecord/coco_val.record*"
  }
  load_instance_masks: true
  mask_type: PNG_MASKS
}

执行训练命令：

python object_detection/model_main_tf2.py ^
--pipeline_config_path=C:/ObjectDetectionAPI/mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8/mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8.config ^
--model_dir=C:/TensoFlow/training_process_2 ^
--alsologtostderr

• 4. 执行验证命令：

python object_detection/model_main_tf2.py ^
--pipeline_config_path=C:/ObjectDetectionAPI/mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8/mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8.config ^
--model_dir=C:/TensoFlow/training_process_2 ^
--checkpoint_dir=C:/TensoFlow/training_process_2 ^
--sample_1_of_n_eval_examples=1 ^
--alsologtostderr

• 5. 导出模型：

python object_detection/exporter_main_v2.py ^
--input_type="image_tensor" ^
--pipeline_config_path=C:/ObjectDetectionAPI/mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8/mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8.config ^
--trained_checkpoint_dir=C:/TensoFlow/training_process_2 ^
--output_directory=C:/TensoFlow/training_process_2/generatedModel

• 6. 转换为TensorFlowJS模型：

tensorflowjs_converter ^
--input_format=tf_saved_model ^
--output_format=tfjs_graph_model  ^
--signature_name=serving_default  ^
--saved_model_tags=serve ^
C:/TensoFlow/training_process_2/generatedModel/saved_model C:/TensoFlow/training_process_2/generatedModel/jsmodel

• 7. 在Angular项目中加载模型，安装依赖：

npm install @tensorflow/tfjs 

加载代码：

ngAfterViewInit() {
   tf.loadGraphModel('/assets/tfmodel/model1/model.json').then((model) => {
     this.model = model;
     this.model.executeAsync(tf.zeros([1, 256, 256, 3])).then((result) => {
       this.loadeModel = true;
     });
   });
}

报错信息 #

tf.min.js:17 ERROR Error: Uncaught (in promise): Error: The dtype of dict['input_tensor'] provided in model.execute(dict) must be int32, but was float32
Error: The dtype of dict['input_tensor'] provided in model.execute(dict) must be int32, but was float32
at F$ (util_base.js:153:11)
at graph_executor.js:721:9
at Array.forEach (<anonymous>)
at e.value (graph_executor.js:705:25)
at e.<anonymous> (graph_executor.js:467:12)
at h (tf.min.js:17:2100)
at Generator.<anonymous> (tf.min.js:17:3441)
at Generator.next (tf.min.js:17:2463)
at u (tf.min.js:17:8324)
at o (tf.min.js:17:8527)
at resolvePromise (zone.js:1211:31)
at resolvePromise (zone.js:1165:17)
at zone.js:1278:17
at _ZoneDelegate.invokeTask (zone.js:406:31)
at Object.onInvokeTask (core.mjs:26343:33)
at _ZoneDelegate.invokeTask (zone.js:405:60)
at Zone.runTask (zone.js:178:47)
at drainMicroTaskQueue (zone.js:585:35)

解决方案 #

1. 调整输入张量的数据类型 #

模型导出时指定的input_type为image_tensor，原始Mask R-CNN的image_tensor输入期望int32类型的0-255像素值，但tf.zeros([1,256,256,3])生成的是float32类型张量。修改加载代码，将输入转换为int32，同时保持与训练一致的1024x1024尺寸：

ngAfterViewInit() {
   tf.loadGraphModel('/assets/tfmodel/model1/model.json').then((model) => {
     this.model = model;
     // 生成int32类型的零张量，实际使用时需将图像数据转为int32
     this.model.executeAsync(tf.zeros([1, 1024, 1024, 3], 'int32')).then((result) => {
       this.loadeModel = true;
     });
   });
}

2. 导出模型时添加预处理逻辑 #

若希望模型接受float32输入（如归一化后的0-1或-1到1范围），可修改pipeline配置文件，在image_resizer后添加归一化预处理，确保输入被转换为模型兼容的类型，再重新导出模型。

3. 确认模型输入签名 #

用TensorFlow的saved_model_cli工具查看模型输入输出签名，验证input_tensor的 dtype 要求：

saved_model_cli show --dir C:/TensoFlow/training_process_2/generatedModel/saved_model --all

根据输出的serving_default签名信息，调整前端输入的类型和尺寸。

内容的提问来源于stack exchange，提问作者Hozeis