使用Hugging Face训练AI时遭遇NoCredentialsError问题求助
问题
我平时很少用Hugging Face,最近尝试在本地机器上结合Hugging Face框架和Amazon SageMaker训练AI,但运行训练代码时一直弹出botocore.exceptions.NoCredentialsError: Unable to locate credentials错误,附上完整报错堆栈和训练代码,求解决办法。
报错信息
PS C:\Users\gboss\OneDrive\Bureau\Ai training> & C:/Users/gboss/AppData/Local/Programs/Python/Python310/python.exe "c:/Users/gboss/OneDrive/Bureau/Ai training/AiTraining.py" Traceback (most recent call last): File "c:\Users\gboss\OneDrive\Bureau\Ai training\AiTraining.py", line 8, in <module> role = iam_client.get_role(RoleName='{IAM_ROLE_WITH_SAGEMAKER_PERMISSIONS}')['Role']['Arn'] File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\client.py", line 514, in _api_call return self._make_api_call(operation_name, kwargs) File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\client.py", line 921, in _make_api_call http, parsed_response = self._make_request( File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\client.py", line 944, in _make_request return self._endpoint.make_request(operation_model, request_dict) File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\endpoint.py", line 119, in make_request return self._send_request(request_dict, operation_model) File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\endpoint.py", line 198, in _send_request request = self.create_request(request_dict, operation_model) File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\endpoint.py", line 134, in create_request self._event_emitter.emit( File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\hooks.py", line 412, in emit return self._emitter.emit(aliased_event_name, **kwargs) File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\hooks.py", line 256, in emit return self._emit(event_name, kwargs) File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\hooks.py", line 239, in _emit response = handler(**kwargs) File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\signers.py", line 105, in handler return self.sign(operation_name, request) File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\signers.py", line 189, in sign auth.add_auth(request) File "C:\Users\gboss\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\auth.py", line 418, in add_auth raise NoCredentialsError() botocore.exceptions.NoCredentialsError: Unable to locate credentials
训练代码
import sagemaker import boto3 from sagemaker.huggingface import HuggingFace # 获取训练作业执行角色 iam_client = boto3.client('iam') role = iam_client.get_role(RoleName='{IAM_ROLE_WITH_SAGEMAKER_PERMISSIONS}')['Role']['Arn'] hyperparameters = { 'model_name_or_path':'ZipperXYZ/DialoGPT-medium-TheWorldMachineExpressive2', 'output_dir':'/opt/ml/model' # 添加剩余超参数 } # Git配置,用于下载微调脚本 git_config = {'repo': 'https://github.com/huggingface/transformers.git','branch': 'v4.17.0'} # 创建Hugging Face estimator huggingface_estimator = HuggingFace( entry_point='run_clm.py', source_dir='./examples/pytorch/language-modeling', instance_type='ml.p3.2xlarge', instance_count=1, role=role, git_config=git_config, transformers_version='4.17.0', pytorch_version='1.10.2', py_version='py38', hyperparameters = hyperparameters ) # 启动训练作业 huggingface_estimator.fit()
解决办法
- 配置本地AWS凭证:boto3无法找到你的AWS身份凭证,需要在本地完成配置:
- 安装AWS CLI后,执行
aws configure命令,按提示输入你的AWS Access Key ID、Secret Access Key、默认区域(如us-east-1)和输出格式。 - 也可以手动在用户目录下创建
.aws文件夹,新建credentials文件,内容格式如下:
同时新建[default] aws_access_key_id = 你的Access Key aws_secret_access_key = 你的Secret Keyconfig文件设置默认区域:[default] region = 你的AWS区域
- 安装AWS CLI后,执行
- 替换占位符角色名:代码中的
{IAM_ROLE_WITH_SAGEMAKER_PERMISSIONS}是占位符,必须替换为你在AWS IAM中实际创建的、拥有SageMaker权限的角色名称。 - 验证权限有效性:确保你配置的AWS账号有权限访问指定的IAM角色,且该角色已经附加了SageMaker相关权限策略(比如AmazonSageMakerFullAccess)。
内容的提问来源于stack exchange,提问作者ZipperXYZ
相关产品推荐
相关产品推荐

