如何通过SageMaker Pipeline部署Hugging Face模型并集成代码？

你提供的单步部署代码如下：

from sagemaker.huggingface import HuggingFaceModel
import sagemaker

role = sagemaker.get_execution_role()

# Hub Model configuration
hub = {
    'HF_MODEL_ID':'siebert/sentiment-roberta-large-english',
    'HF_TASK':'text-classification'
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    transformers_version='4.17.0',
    pytorch_version='1.10.2',
    py_version='py38',
    env=hub,
    role=role, 
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
    initial_instance_count=1,
    instance_type='ml.g4dn.xlarge'
)

要将这段逻辑集成到SageMaker Pipeline，需要把单步操作拆解为Pipeline的可编排步骤，以下是具体实现方法：

1. 核心依赖与组件 #

需要导入SageMaker Pipeline的核心模块，包括用于编排步骤的Pipeline、封装模型创建的ModelStep，以及部署模型的DeployModelStep。

2. 完整Pipeline集成代码 #

import sagemaker
from sagemaker.huggingface import HuggingFaceModel
from sagemaker.workflow.pipeline import Pipeline
from sagemaker.workflow.model_step import ModelStep
from sagemaker.workflow.steps import DeployModelStep

# 获取执行角色
role = sagemaker.get_execution_role()

# Hugging Face Hub模型配置
hub_config = {
    'HF_MODEL_ID': 'siebert/sentiment-roberta-large-english',
    'HF_TASK': 'text-classification'
}

# 1. 定义Hugging Face模型
hf_model = HuggingFaceModel(
    transformers_version='4.17.0',
    pytorch_version='1.10.2',
    py_version='py38',
    env=hub_config,
    role=role,
)

# 2. 创建模型步骤（封装模型定义）
model_step = ModelStep(
    name="Create-HuggingFace-Model",
    step_args=hf_model.create(instance_type="ml.m5.large")  # 指定用于模型创建的实例类型
)

# 3. 创建部署步骤
deploy_step = DeployModelStep(
    name="Deploy-HuggingFace-Model",
    model=model_step.properties.ModelName,
    initial_instance_count=1,
    instance_type="ml.g4dn.xlarge",
)

# 4. 构建Pipeline
pipeline = Pipeline(
    name="HuggingFace-Sentiment-Pipeline",
    steps=[model_step, deploy_step],
)

# 5. 提交并运行Pipeline
pipeline.upsert(role_arn=role)
execution = pipeline.start()

# 查看运行状态
execution.describe()

3. 关键说明 #

• ModelStep：负责在Pipeline中实例化HuggingFace模型，需指定用于模型创建的实例类型（通常用低成本的ml.m5系列即可）。
• DeployModelStep：定义模型部署的实例配置，参数和单步部署对应，但通过Pipeline属性引用模型名称。
• Pipeline运行：通过upsert创建或更新Pipeline，start触发执行，可通过execution.describe()跟踪运行状态。

可选：添加模型注册步骤 #

如果需要将模型注册到SageMaker模型注册表，可新增RegisterModelStep：

from sagemaker.workflow.step_collections import RegisterModel

# 定义模型注册步骤
register_step = RegisterModel(
    name="Register-HuggingFace-Model",
    model=model_step.properties.ModelName,
    content_types=["application/json"],
    response_types=["application/json"],
    inference_instances=["ml.g4dn.xlarge"],
    transform_instances=["ml.m5.large"],
    model_package_group_name="huggingface-sentiment-group",
)

# 更新Pipeline步骤顺序
pipeline = Pipeline(
    name="HuggingFace-Sentiment-Pipeline",
    steps=[model_step, register_step, deploy_step],
)

内容的提问来源于stack exchange，提问作者Soumya