在AWS SageMaker运行TensorFlow 2.x模型推理时遇输入张量未找到错误

我基于MobileNetV2训练了一个可对苹果、胡萝卜、黄瓜、梨、西葫芦进行分类的模型，将.h5模型转换为protobuf格式后部署到AWS SageMaker端点，但调用推理时收到错误提示：Tensor input_1:0, specified in either feed_devices or fetch_devices was not found in the Graph，我检查protobuf文件确认输入张量确实为input_1:0，不清楚错误来源。

推理代码 #

from tensorflow.keras.preprocessing import image
import numpy as np
import json
import boto3

endpoint_name = 'tensorflow-inference-2023-06-15-03-24-26-323'
label_names = ['apple', 'carrot', 'cucumber', 'pear', 'zucchini']

runtime = boto3.client('runtime.sagemaker')

img = image.load_img("image-removebg-preview (4).png", target_size = (200,200))
x = image.img_to_array(img)
x = np.expand_dims(x, axis = 0)
x=x/255.0

payload = {"instances": x.tolist()}

# Invoke the endpoint
response = runtime.invoke_endpoint(EndpointName=endpoint_name, ContentType='application/json', Body=json.dumps(payload))

# Parse the response
result = json.loads(response['Body'].read().decode())
predictions = result['predictions'][0]

# Print the predicted label
predicted_label_index = np.argmax(predictions)
predicted_label = label_names[predicted_label_index]
print(predicted_label)

错误栈 #

ModelError                                Traceback (most recent call last)
Cell In[6], line 19
     16 payload = {"instances": x.tolist()}
     18 # Invoke the endpoint
---> 19 response = runtime.invoke_endpoint(EndpointName=endpoint_name, ContentType='application/json', Body=json.dumps(payload))
     21 # Parse the response
     22 result = json.loads(response['Body'].read().decode())

File ~/anaconda3/envs/tensorflow2_p310/lib/python3.10/site-packages/botocore/client.py:530, in ClientCreator._create_api_method.<locals>._api_call(self, *args, **kwargs)
    526     raise TypeError(
    527         f"{py_operation_name}() only accepts keyword arguments."
    528     )
    529 # The "self" in this scope is referring to the BaseClient.
--> 530 return self._make_api_call(operation_name, kwargs)

File ~/anaconda3/envs/tensorflow2_p310/lib/python3.10/site-packages/botocore/client.py:964, in BaseClient._make_api_call(self, operation_name, api_params)
    962     error_code = parsed_response.get("Error", {}).get("Code")
    963     error_class = self.exceptions.from_code(error_code)
--> 964     raise error_class(parsed_response, operation_name)
    965 else:
    966     return parsed_response

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
    "error": "Tensor input_1:0, specified in either feed_devices or fetch_devices was not found in the Graph"
}".

可能的原因及解决方法 #

1. 模型转换未固化输入输出签名 #

SageMaker的TensorFlow推理容器依赖模型签名解析输入输出，若转换时未显式指定签名，会导致张量匹配失败。

解决步骤：
用tf.saved_model.save重新保存模型，显式定义输入输出签名：

import tensorflow as tf
from tensorflow.keras.models import load_model

model = load_model('your_model.h5')
# 匹配你的模型输入形状，name设为input_1
input_signature = tf.TensorSpec(shape=(None, 200, 200, 3), dtype=tf.float32, name='input_1')
# 替换为你的输出层名称和形状
output_signature = tf.TensorSpec(shape=(None, 5), dtype=tf.float32, name='dense_1')

@tf.function(input_signature=[input_signature])
def serving_fn(inputs):
    return {'predictions': model(inputs)}

# 保存为符合SageMaker要求的格式
tf.saved_model.save(model, 'saved_model', signatures={'serving_default': serving_fn})

将生成的saved_model文件夹打包上传，重新创建端点。

2. 请求payload格式不兼容 #

部分TensorFlow推理容器对instances格式支持存在差异，可尝试替换为inputs格式：

payload = {"inputs": x.tolist()}

3. 推理容器版本不匹配 #

若训练使用的TensorFlow版本与SageMaker容器版本差异过大，会导致模型加载异常。

解决：
创建端点时选择与训练版本一致的容器，例如训练用TensorFlow 2.10，就选择对应区域的2.10-cpu或2.10-gpu容器。

4. 模型文件夹结构不符合要求 #

SageMaker要求TensorFlow模型的文件夹结构必须为：

model/
└── 1/
    ├── saved_model.pb
    └── variables/
        ├── variables.data-00000-of-00001
        └── variables.index

检查并调整结构后重新上传部署。

内容的提问来源于stack exchange，提问作者Rishi Athavale