在SageMaker部署HuggingFace模型时遭遇500服务器错误求助

我已在SageMaker Studio中用自有数据成功微调sentence-transformers的all-MiniLM-L12-v2模型，模型保存至S3，文件名为model.tar.gz。为部署该模型做推理，我按照HuggingFace文档要求添加自定义推理模块：下载并解压model.tar.gz，添加code/inference.py后重新打包上传至S3。端点创建成功，但调用predictor.predict()时直接崩溃，报错如下：

报错详情 #

InvokeEndpoint报错 #

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (500) from primary with message "{
  "code": 500,
  "type": "InternalServerException",
  "message": "Worker died."
}

CloudWatch日志警告 #

2024-07-30T13:19:09,702 [WARN ] W-9000-model_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: model, error: Worker died.

端点创建代码 #

from sagemaker.huggingface.model import HuggingFaceModel
from sagemaker import get_execution_role, image_uris

role            = get_execution_role()
estimator_image = image_uris.retrieve(framework='pytorch',region='eu-west-1',version='2.0.0',py_version='py310',image_scope='inference', instance_type='ml.g5.4xlarge')
sm_model_ref    = model_path

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    model_data    = sm_model_ref,
    role          = role,                                                     
    image_uri     = estimator_image,
)

自定义inference.py代码 #

文件路径为model.tar.gz/code/inference.py，代码内容：

import torch

# Create a custom inference to overwrite the default method
def predict_fn(data, model):

    # create sentences pair
    sentences1 = data["premise"]
    sentences2 = data["hypothesis"]
 
    # Compute token embeddings
    with torch.no_grad():
        embeddings1 = model.encode(sentences1)
        embeddings2 = model.encode(sentences2)
        
        # Compute cosine similarities        
        similarities = model.similarity(embeddings1, embeddings2)
 
    return similarities

模型文件结构 #

model.tar.gz
 |_ _ 1_Pooling
 |_ _ 2_Normalize
 |_ _ checkpoint-8300
 |_ _ checkpoint-8334
 |_ _ code
   |_ _ inference.py
 |_ _ config_sentence_transformers.json
 |_ _ config.json
 |_ _ model.safetensors
 |_ _ module.json
 |_ _ README.md
 |_ _ sentence_bert_config.json
 |_ _ special_token_map.json
 |_ _ tokenizer_config.json
 |_ _ tokenizer.json
 |_ _ vocab.txt

内容的提问来源于stack exchange，提问作者Yoan B. M.Sc