如何通过Azure机器学习注册表下载HuggingFace模型？遇URI解析错误

我能正常从自有Azure Machine Learning注册表甚至"azureml"注册表下载任意模型，但用相同代码访问HuggingFace注册表时，收到错误Exception: Registry asset URI could not be parsed。

复现步骤（Azure计算实例） #

registry_name = "HuggingFace"

from azure.ai.ml import MLClient
ml_client_registry = MLClient(credential=credential, registry_name=registry_name)
m_name    = "openai-gpt"
m_version = 12

m = ml_client_registry.models.get(name=m_name, version=m_version)

m_local_base_path = "./models_from_huggings_registry"

ml_client_registry.models.download(name=m_name, version=m_version, download_path=m_local_base_path)

模型元数据（打印m变量） #

Model({'job_name': None, 'is_anonymous': False,
'auto_increment_version': False, 'name': 'openai-gpt', 'description':
'openai-gpt是Hugging Face Hub上的预训练语言模型，专为transformers库中的text-generation任务设计。你可以在模型的专属Model Card上了解更多架构、超参数、限制和偏差信息。

以下是用于获取模型预测的API请求示例：

{
  "inputs": "My name is Julien and I like to"
}

', 'tags': {'modelId': 'openai-gpt', 'task':
'text-generation', 'library': 'transformers', 'license': 'mit'},
'properties': {'skuBasedEngineIds':
'azureml://registries/HuggingFace/models/transformers-cpu-small/labels/latest,azureml://registries/HuggingFace/models/transformers-gpu-medium/labels/latest',
'engineEnvironmentVariableOverrides': '{"AZUREML_HF_MODEL_ID":
"openai-gpt", "AZUREML_HF_TASK": "text-generation"}'},
'print_as_yaml': True, 'id':
'azureml://registries/HuggingFace/models/openai-gpt/versions/12',
'Resource__source_path': None, 'base_path':
'/mnt/batch/tasks/shared/LS_root/mounts/clusters/dsvm-general-optimized01/code/Users/mauro.minella/git_repos/azuremlnotebooks/MLOPS/notebooks
AMLv2', 'creation_context':
<azure.ai.ml.entities._system_data.SystemData object at
0x7f2602efdf60>, 'serialize': <msrest.serialization.Serializer object
at 0x7f25bf52c130>, 'version': '12', 'latest_version': None, 'path':
None, 'datastore': None, 'utc_time_created': None, 'flavors': None,
'arm_type': 'model_version', 'type': 'preset_model'})

完整错误栈 #

TypeError                                 Traceback (most recent call last)
File /anaconda/envs/azuremlsdkv2mm/lib/python3.10/site-packages/azure/ai/ml/_utils/_storage_utils.py:187, in get_ds_name_and_path_prefix(asset_uri, registry_name)
    186 try:
---> 187     split_paths = re.findall(STORAGE_URI_REGEX, asset_uri)
    188     path_prefix = split_paths[0][3]

File /anaconda/envs/azuremlsdkv2mm/lib/python3.10/re.py:240, in findall(pattern, string, flags)
    233 """Return a list of all non-overlapping matches in the string.
    234 
    235 If one or more capturing groups are present in the pattern, return
   (...)
    238 
    239 Empty matches are included in the result."""
---> 240 return _compile(pattern, flags).findall(string)

TypeError: expected string or bytes-like object

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
Cell In[21], line 6
      2 import mlflow
      4 m_local_base_path = "./models_from_huggings_registry"
----> 6 ml_client_registry.models.download(name=m_name, version=m_version, download_path=m_local_base_path)

File /anaconda/envs/azuremlsdkv2mm/lib/python3.10/site-packages/azure/ai/ml/_telemetry/activity.py:263, in monitor_with_activity.<locals>.monitor.<locals>.wrapper(*args, **kwargs)
    260 @functools.wraps(f)
    261 def wrapper(*args, **kwargs):
    262     with log_activity(logger, activity_name or f.__name__, activity_type, custom_dimensions):
---> 263         return f(*args, **kwargs)

File /anaconda/envs/azuremlsdkv2mm/lib/python3.10/site-packages/azure/ai/ml/operations/_model_operations.py:305, in ModelOperations.download(self, name, version, download_path)
    295 """Download files related to a model.
    296 
    297 :param str name: Name of the model.
   (...)
    301 :raise: ResourceNotFoundError if can't find a model matching provided name.
    302 """
    304 model_uri = self.get(name=name, version=version).path
---> 305 ds_name, path_prefix = get_ds_name_and_path_prefix(model_uri, self._registry_name)
    306 if self._registry_name:
    307     sas_uri = get_storage_details_for_registry_assets(
    308         service_client=self._service_client,
    309         asset_name=name,
   (...)
    314         uri=model_uri,
    315     )

File /anaconda/envs/azuremlsdkv2mm/lib/python3.10/site-packages/azure/ai/ml/_utils/_storage_utils.py:190, in get_ds_name_and_path_prefix(asset_uri, registry_name)
    188         path_prefix = split_paths[0][3]
    189     except Exception:
---> 190         raise Exception("Registry asset URI could not be parsed.")
    191     ds_name = None
    192 else:

Exception: Registry asset URI could not be parsed.

原因分析 #

从模型元数据可以看到，HuggingFace注册表中的这类模型类型是preset_model（预设模型），其path字段为None，说明这些模型并没有实际存储在Azure的存储服务中，而是指向Hugging Face Hub的模型条目。download方法仅适用于存储在Azure存储中的模型资产，所以调用该方法会因为无法解析有效存储URI而报错。

解决办法 #

1. 直接部署使用：这类预设模型是为Azure ML部署优化的，你可以直接通过Azure ML将其部署为在线端点或批量端点，无需本地下载。
2. 使用Hugging Face库下载：如果需要本地使用模型文件，直接使用transformers库的from_pretrained方法下载：

from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("openai-gpt")
tokenizer = AutoTokenizer.from_pretrained("openai-gpt")
# 保存到本地
model.save_pretrained("./models_from_huggings_registry")
tokenizer.save_pretrained("./models_from_huggings_registry")

内容的提问来源于stack exchange，提问作者Mauro Minella